reece@enuxha.eas.asu.edu (Glen A. Reece) (06/16/90)
I decided to go ahead and post the summary of the replies that I received regarding the 32K barrier on the Mac. This is a simple compilation of the replies that I recieved, and I have not verified whether they are "truely" correct (as one of the replies suggested). Again, thanks to all who replied!!! - Glen A. Reece Industrial Fellow Artificial Intelligence Lab. Arizona State University ==================================================================== ==================================================================== From norman@d.cs.okstate.edu Sun Jun 3 09:10:58 1990 Received: from [192.31.83.65] by enuxha.eas.asu.edu (5.61/1.28) id AA00378; Sun, 3 Jun 90 09:10:51 -0700 Message-Id: <9006031610.AA00378@enuxha.eas.asu.edu> Date: Sun, 3 Jun 90 11:11:33 CDT From: Norman Graham Oklahoma State Univers <norman@d.cs.okstate.edu> To: reece@enuxha.eas.asu.edu Status: R Subject: Re: 32K barrier on the Mac... Reply-To: norman@a.cs.okstate.edu cc: norman@a.cs.okstate.edu I'm working from memory, so I may munge minor details. Here goes... The 32K globals limit is _not_ imposed by the Mac's architecture. Rather, it is a limitation imposed by the method many development systems use to reference global data. Reasonable development systems will not have this limit (for example, MPW C, Pascal, and Link can build applications with > 32K globals). The global referencing method that produces the 32K limit works like this: The Mac can load applications at any point in memory, so absolute memory references are out; thus globals must be referenced relative to some fixed position in memory. The Mac's Segment Loader automatically sets up register A5 to point to the bottom of the app's stack, so many development systems take advantage of this fixed point in memory and store the application globals on the negative side of the stack's base. To reference the globals, the development system will use the 'address register indirect with displacement' addressing mode with a negative displacement [in asm it looks like -32(A5) ]. With this addressing mode, the displacement is limited to a signed 16-bit value (on the 68000 at least) and thus the largest negative displacement is -32K. Hence the 32K limit on globals. The preceeding method is but one of several methods that can be used to reference global data. Most development systems (particularly the early ones) use this method because (1) most mac applications don't come anywhere close to using 32K of globals and (2) it works, it's relativly fast, and it saves a register (since A5 is already in use for other things). In MPW, compiler and link options can be used to generate 32-bit references, but the resulting code is slower--but only for the code that uses the 32-bit global references; code that uses 16-bit global references are still fast. The moral of the story is 'If you need > 32K globals then use a development system that can handle it.' Hope this helps, Norm ======================================================================== From philip@Pescadero.Stanford.EDU Sun Jun 3 12:42:36 1990 Received: from Pescadero.Stanford.EDU by enuxha.eas.asu.edu (5.61/1.28) id AA01867; Sun, 3 Jun 90 12:42:34 -0700 Received: from LOCALHOST by Pescadero.Stanford.EDU (5.59/25-eef) id AA01051; Sun, 3 Jun 90 12:44:04 PDT Message-Id: <9006031944.AA01051@Pescadero.Stanford.EDU> To: reece@enuxha.eas.asu.edu Subject: Re: 32K barrier on the Mac... Date: Sun, 03 Jun 90 12:43:59 PDT From: philip@Pescadero.Stanford.EDU Status: R In article <894@enuxha.eas.asu.edu> you write: > I was having a discussion the other day with a friend who asked about > writing applications for the Mac. I told him that the limitation to > application size was not memory but the 32K static data barrier that exists > on the Mac (64K on the PC I think). Anyway, can someone please give a > good explanation of the reason for the 32K limit and reason as to why Apple > has not eliminated this constraint. To start with, the Mac OS does restrict the maximum amount of usable memory (8M in current versions, I believe - I've been using 5M for 2 years with no problem). There are in fact several variants on the "32K" limit. The one you refer to - the maximum for static data - is not really Apple's "fault", except to the extent that they may supply compilers which are restricited in this way. The MC680x0 instruction set includes instructions which allow efficient addressing within an offset of 32K, and most compiler writers have chosen to enforce the 32K limit, rather than to have to generate different instructions for specific pieces of data more than 32K from the start of the global area. A few other limits... What follows is from memory, so if you want to post this or use the information, check it from other sources. [The built-in text edit routines have a size limit (usually assumed to be 32K) - this results from a length field being stored as an integer. In fact, according to Inside Macintosh, the length field is "unsigned", so the limit ought to be 64K (but I haven't checked this, and every Mac programmming book I've seen says 32K). Another is the limit on size of code resources, which I believe is also 32K, at least I think the entry point furthest from the start can't be more than 32K from the begining because of the way the jump table (used for calls across segments) is structured. Finally, the earlier Macs had a limit of 32K on the size of resources, but I believe this has been fixed with later ROM revisions and newer models (this was I believe a bug in the resource manager).] The 64K limit on the PC is a consequence of the segment addressing scheme on the 8086 processor, which restricts a segment to 64K. I'm not sure about the limit on the 80286, but the 80386 and 80386SX allow a single segment register to be used to access the entire address space. Compared with the 8086, the 68000 does not introduce an addressing limitation, but allows an optimization for offsets less than 32K. Also, the 8086 has too few segment registers, which means a vast amount of register copying is required in some cases (e.g., a context switch). In summary, the Mac addressing limits are software-based, and could be repaired (sometimes at the expense of making the compiler writer do more work to generate optimal code); the 8086 addressing limits are hardware based, and can only be cured by moving to a more expensive processor. ========================================================================== From well!oster@apple.com Mon Jun 4 05:49:08 1990 Received: from apple.com by enuxha.eas.asu.edu (5.61/1.28) id AA09948; Mon, 4 Jun 90 05:49:05 -0700 Received: by apple.com (5.61/25-eef) id AA19665; Mon, 4 Jun 90 05:29:19 -0700 for Received: by well.sf.ca.us (4.12/4.7) id AA11892; Mon, 4 Jun 90 01:06:26 pdt Date: Mon, 4 Jun 90 01:06:26 pdt From: well!oster@apple.com (David Phillip Oster) Message-Id: <9006040806.AA11892@well.sf.ca.us> To: reece@enuxha.eas.asu.edu Subject: Re: 32K barrier on the Mac Status: R There is and is not any such limit. Many compilers use signed 16-bit offsets from register A5 to access static data, and those compilers will not let you directly generate code to access more than 32k of statics. If you have a large static structure, you can always write it to a Macintosh Resource, and have the initialization section of the program do a: h = GetResource('GNRL', 128); MoveHHi(h); HLock(h); basePointer = *h; and access things as basePointer->field[i] from then on. You'd have to do this anyway if you prgrammed in pascal: Resources are the _only_ kind of initialized data there are, since ordinary pascal lacks the concept. Many compilers do not allow more than 32k in a single struct. I have been writing for the Mac since it came out, and for Unix since '75. I haven't found the restrictions a hardship. ========================================================================== From rmh@apple.com Mon Jun 4 10:23:14 1990 Received: from apple.com by enuxha.eas.asu.edu (5.61/1.28) id AA13438; Mon, 4 Jun 90 10:23:11 -0700 Received: from [90.1.0.10] by apple.com with SMTP (5.61/25-eef) id AA02558; Mon, 4 Jun 90 10:25:15 -0700 for reece@enuxha.eas.asu.edu Received: by goofy.apple.com with SMTP (5.61/25-eef) id AA08199; Mon, 4 Jun 90 10:24:56 -0700 for rmh@apple.com From: rmh@apple.com (Rick Holzgrafe) To: reece@enuxha.eas.asu.edu (Glen A. Reece) Cc: rmh@apple.com Subject: Re: 32K barrier on the Mac... Date: Mon, 4 Jun 1990 10:19:24 PDT References: <894@enuxha.eas.asu.edu> Message-Id: <85449006041019rmh@apple.com> Organization: Apple Computer, Inc. Status: R In article <894@enuxha.eas.asu.edu> reece@enuxha.eas.asu.edu (Glen A. Reece) writes: > I was having a discussion the other day with a friend who asked about > writing applications for the Mac. I told him that the limitation to > application size was not memory but the 32K static data barrier that exists > on the Mac (64K on the PC I think). Anyway, can someone please give a > good explanation of the reason for the 32K limit and reason as to why Apple > has not eliminated this constraint. This constraint exists but can be avoided. For code, it is most convenient if your code segments do not exceed 32K. This is because most compilers and linkers use 16-bit offsets for their JSR instructions within segments, and because the "jump table" which controls jumps between segments uses 16-bit offsets to index into a segment. There are two ways around this. One is to simply use multiple segments, all less than 32K in size. There is rarely any difficulty with this in a standard application, and all Mac development systems support easy code segmenting. (Not all code resides in applications. Desk accessories, drivers, and INITs are examples of code which is "supposed" to be all in one segment; however, even these can have multiple code segments with only small additional effort from the developer.) The second way around is that some linkers (e.g. MPW's) allow code segments > 32K, and permit arranging the code within the segment so that external entry points (those accessed from outside the segment, via the jump table) are all located in the first 32K. A similar limitation holds for static data. Mac applications have a single segment devoted to global data, and this segment must be < 32K in size, again because of the use of 16-bit indexing. This too can be gotten around. For example, large arrays can be represented instead as a pointer. The pointer uses only four bytes of the data segment. At run time, you dynamically allocate the array space from the heap, and aim the pointer at it. There is no arbitrary limit to the heap size or the size of any chunk allocated from the heap: all of available memory is usable. High-end image processing applications typically work with memory chunks > 1Meg, for example. If you require a large *initialized* array, you can still use the method above. Instead of allocating uninitialized space from the heap, store the contents of the array as a resource (at build time), read that into the heap at run time, and aim the pointer at it. (The MPW tool "Rez" reads a source language similar to, but richer than, C for declaring initialized data structures and translating them into resources.) Segmenting code, allocating space from the heap, and storing data as resources are techniques perhaps likely to sound difficult and kludgy to the non-Mac programmer. In fact they are standard parts of the Mac programming idiom, and are not difficult at all. All of the above techniques are accomplished with not more than four lines of code, including error handling for the out-of-memory condition. Segmenting involves only instructions to the compiler/linker, and is transparent to the running code. Usually the only people inconvenienced by the 32K "limit" are folks who are porting code from other systems (like Unix), and who have either large arrays of compile-time-initialized data, or single huge sub-routines that are themselves > 32K in size. Such programs can't be ported with a simple re-compile; they must suffer some re-arrangement of the source. But for anyone writing Mac applications from scratch, the 32K thing is not a problem. Hope this helps. ========================================================================== Rick Holzgrafe | {sun,voder,nsc,mtxinu,dual}!apple!rmh Software Engineer | AppleLink HOLZGRAFE1 rmh@apple.com Apple Computer, Inc. | "All opinions expressed are mine, and do 20525 Mariani Ave. MS: 77-A | not necessarily represent those of my Cupertino, CA 95014 | employer, Apple Computer Inc." ========================================================================== From coherent!dplatt@ames.arc.nasa.gov Mon Jun 4 11:05:46 1990 Received: from ames.arc.nasa.gov by enuxha.eas.asu.edu (5.61/1.28) id AA14775; Mon, 4 Jun 90 11:05:43 -0700 Received: by ames.arc.nasa.gov (5.61/1.2); Mon, 4 Jun 90 11:07:50 -0700 Received: from improper.coherent.com by coherent.com (4.1/SMI-3.2) id AA23694; Mon, 4 Jun 90 11:02:28 PDT Received: by improper.coherent.com (4.0/SMI-3.2) id AA09021; Mon, 4 Jun 90 11:02:25 PDT Date: Mon, 4 Jun 90 11:02:25 PDT From: dplatt@coherent.com Message-Id: <9006041802.AA09021@improper.coherent.com> To: reece@enuxha.eas.asu.edu Subject: Re: 32K barrier on the Mac... Newsgroups: comp.sys.mac In-Reply-To: <894@enuxha.eas.asu.edu> Organization: Coherent Thought Inc., Palo Alto CA Status: R There are several ways in which a 32k limit of one sort or another can creep in. Here they are: 1) In the first versions of the Mac (the 128 and 512k machines with the 64k ROM), individual resources in the application couldn't exceed 32k (or so I recall). This limitation does not exist in newer Macs, and is no longer relevant. 2) Most applications access their static data (globals) by using an addressing format which specifies the base of the globals (stored in register A5) and a signed offset (16 bits). The use of a 16-bit signed offset limits the access to +/- 32k from the location to which A5 is pointing. Hence, this addressing format cannot access a globals area larger than 32k bytes. Some development systems (e.g. MPW) now support compiler options which specify that access to the global area should use a longword offset (32 bits) rather than a short offset (16 bits). Applications built using these development systems can have very large static data pools. 3) Access to Macintosh subroutines is performed via the "jump table". When you call a subroutine, you actually branch to an entry in the jump table. A jump-table entry starts out containing a set of instructions which load a segment number and a 16-bit offset, and then executes a code-loading trap; once the segment has been loaded into memory, the jump-table entry is replaced by a direct branch to the real subroutine. Because the jump table contains a 16-bit signed offset, any subroutine you wish to call must begin with 32k of the beginning of its segment. Subroutines which are called only from within an individual segment (private, intra-segment routines) can safely exist beyond the 32k boundary; the compiler can use a PC-relative jump to call them, and is not required to go through the jump table. 4) The jump table itself is stored in the same way as the application globals... it's stored just below the location referenced by register A5, while the applications are stored above A5 (or vice versa... I forget which is which). For this reason, the total size of the jump table itself must be less than 32k bytes; this limits the number of individual globally-accessible subroutines which can exist in a single application. For applications coded in C or PASCAL, this isn't usually a problem. Large object-oriented systems, with a large number of classes and methods, may run up against the jump-table-size limit. -- Dave Platt VOICE: (415) 493-8805 UUCP: ...!{ames,apple,uunet}!coherent!dplatt DOMAIN: dplatt@coherent.com INTERNET: coherent!dplatt@ames.arpa, ...@uunet.uu.net USNAIL: Coherent Thought Inc. 3350 West Bayshore #205 Palo Alto CA 94303 ========================================================================= From brad@gobi.Jpl.Nasa.Gov Wed Jun 6 12:32:32 1990 Received: from elroy.jpl.nasa.gov by enuxha.eas.asu.edu (5.61/1.28) id AA26794; Wed, 6 Jun 90 12:32:12 -0700 Received: from gobi.Jpl.Nasa.Gov (aisl-gw.jpl.nasa.gov) by elroy.Jpl.Nasa.Gov (4.1/SMI-4.0+DXR) id AA28278; Wed, 6 Jun 90 12:34:10 PDT Received: by gobi.Jpl.Nasa.Gov (4.0/SMI-3.4+DXR+sub) id AA09737; Wed, 6 Jun 90 12:33:45 PDT Date: Wed, 6 Jun 90 12:33:45 PDT From: brad@gobi.Jpl.Nasa.Gov (Brad Pickering) Message-Id: <9006061933.AA09737@gobi.Jpl.Nasa.Gov> To: reece@enuxha.eas.asu.edu Subject: Re: 32K barrier on the Mac... Newsgroups: comp.sys.mac In-Reply-To: <894@enuxha.eas.asu.edu> Organization: Jet Propulsion Laboratory, Pasadena, CA Cc: Status: R The natural way for a mac to express relative addresses (the most common type of addressing) is as a 16 bit signed number (approx. -32k to 32k). This is more because of the motorola 68000 than the mac. There are other ways to get around this 32k number but they are not as natuaral. The more recent develoment systems (like MPW) allow you to specify a flag when compiling programs which will use the more unatural addressing method to get around the limit. The programs will run slightly slower but it can be worth it. The motorola 68020 and up have a natural way to use 32 bit signed numbers as addresses, but if you want your program to run on all macs this feature cant be used. Hope this helps, Brad Pickering. ps: the IBM machines express addresses as 16 bit UNsigned numbers (0 - 64k). If this doesn't help then you are welcome to write to me for more info. ======================================================================= From woody@tybalt.caltech.edu Sat Jun 9 00:12:32 1990 Received: from tybalt.caltech.edu by enuxha.eas.asu.edu (5.61/1.28) id AA19432; Sat, 9 Jun 90 00:12:27 -0700 Received: by tybalt.caltech.edu (5.52/1.2) id AA09677; Sat, 9 Jun 90 00:12:59 PDT Date: Sat, 9 Jun 90 00:12:59 PDT From: woody@tybalt.caltech.edu (William Edward Woody) Message-Id: <9006090712.AA09677@tybalt.caltech.edu> To: reece@enuxha.eas.asu.edu Subject: Re: 32K barrier on the Mac... Newsgroups: comp.sys.mac In-Reply-To: <894@enuxha.eas.asu.edu> Organization: California Institute of Technology, Pasadena Cc: Status: RO In article <894@enuxha.eas.asu.edu> you write: >Greetings again from AZ: > > I was having a discussion the other day with a friend who asked about >writing applications for the Mac. I told him that the limitation to >application size was not memory but the 32K static data barrier that exists >on the Mac (64K on the PC I think). Anyway, can someone please give a >good explanation of the reason for the 32K limit and reason as to why Apple >has not eliminated this constraint. > > Please respond to me directly, and if others are interested then I'll >post the responses back to comp.sys.mac > > Thankx in advance, > > - Glen A. Reece > Industrial Fellow > Artificial Intelligence Lab. > Arizona State University Actually, this one's easy. The MC68000 CPU that was originally used in the original Macintosh's allow relative offsets from any address register; the problem was, that the max offset amount was a 16 bit word. This allows offsets from an address register to be up to +/- 32K bytes. Most code generators try to generate a MOVE.x -NN(A5) to load or save data to the static global area (register A5 is reserved as a pointer to global memory space). The version of MPW I am currently running (that is, MPW C v3.1) allows static memory greater than 32K; this is done by generating an explicit MOVEA A5, A0; ADD.L -NN,A0; MOVE (A0) instead of the MOVE.x -NN(A5) on the MC68000. This takes a tiny bit longer to get the value of a globa variable, but does allow more than 32K of static global store. Of course if you are compiling to an MC68020 or 68030 family CPU hardware, the proper MOVE.x -NN.L(A5) [or whatever] instructions are generated, to allow long-word arithmetic against the A5 register. I think Apple did it this way because they (1) wanted the most flexible way of handling global memory possible [globals can land anywhere the OS feels like placing them; this makes Multifinder possible], and (2) they were enamored with the MC68000's machine language addressing modes... ;-) -- -- William Edward Woody -------------------------------------------------------------- Apple File Edit Special Fonts Sizes -------------------------------------------------------------- +--------------------------+ +-+ |=[]== E-Mail Address =====| |S| |--------------------------| | | | woody@tybalt.caltech.edu |-------------------+ +-+ +--------------------------+= Home Address ====| XP_60 +---------------|-----------------------| |=[]== Work Addr| P. O. Box 50986 |---------+ +---------------| Pasadena, CA 91115 |ICBM ====| | JPL 510-202 +-----------------------+---------| | 4800 Oak Grove Dr. | | 34 08' 44" N |+- | Pasadena, CA 91109 | | 118 08' 44" W ||| +-----------------------+ +---------------+|| +---+ Trash -------------------------------------------------------------- Okay, okay, so it's a bit excessive. ======================================================================= = E N D O F F I L E = =======================================================================