[comp.sys.mac.programmer] Think C 32K global data limit: workaround?

mark@intek01.UUCP (Mark McWiggins) (12/15/89)

We're getting discouraged with the speed of MPW C (especially with C++)
and are interested in evaluating Think C as a target for C++ 2.0. 
But the 32K global data limit is a concern.  Has anyone come up with 
a satisfactory workaround?

Thanks in advance.


-- 
Mark McWiggins			Integration Technologies, Inc. (Intek)
+1 206 455 9935			DISCLAIMER:  I could be wrong ...
1400 112th Ave SE #202		Bellevue WA  98004
uunet!intek01!mark		Ask me about C++!

ech@cbnewsk.ATT.COM (ned.horvath) (12/17/89)

From article <257@intek01.UUCP>, by mark@intek01.UUCP (Mark McWiggins):
> We're getting discouraged with the speed of MPW C (especially with C++)
> and are interested in evaluating Think C as a target for C++ 2.0. 
> But the 32K global data limit is a concern.  Has anyone come up with 
> a satisfactory workaround?

This is a classic red herring in Macintosh programming: you aren't limited
to 32K of data space, just to 32K of named globals.  You can allocate any
amount of memory up to available RAM (or user-specified RAM partition under
Multifinder) by using NewPtr or, better, NewHandle.  The difference is

/* lazy, VAX-programmer method */
	int foo[20000];
/* enlightened, Mac programmer method */
	int *foo;
	... foo = NewHandle (20000 * (Size) sizeof *foo);
/* even more enlightened method */
	int *foo;
	/* figure out how big foo NEEDS to be */
	foo = NewHandle (whatzit * (Size) sizeof *foo);

The first method requires ungodly (>>32K) of global data space, and has the
additional disadvantage that it blows up when the problem size is 20001.
Which is probably your third customer.

The second method still breaks for 20001, but requires 4, count 'em 4, bytes
of global data space.  You're still limited to 8000 dynamic arrays, but I'll
bet you can live with that.

The third method reflects good programming practice and also needs only global
space for the pointer.  Oh, yeah: the second and third methods MUST be
followed by
	if (foo == NIL) {
		/* tell the user the bad news */
		/* fail in some nice fashion */
	}

If you need a huge initialized array, put it in a resource.  If you need a
huge, initialized, writeable array, put it in a resource, mark the resource
non-purgable, and DetachResource it so your program will continue to run
when some bright character at Apple figures out how to share resources
over a network.  Oh, yeah: make sure your GetResource succeeds in loading
that nasty array, same sad story and gentle termination in the face of
failure: this is the Mac, not Cleveland, to paraphrase Sam Wyche...

The only serious problem you face is that all extant O-O programming systems
on the Mac store their method-dispatch tables in global dataspace, so there's
the possibility that a very complex class hierarchy will blow you out of the
water.  MPW is no better than LSC in this respect, and until recently ALL
Object Pascal methods went in the (above A5) jump table at 8 bytes a pop,
another painful limit with even a moderatedly complex class structure.
I don't know if that latter situation has been corrected; I know there
was a lot of concern with it at Apple (perhaps Larry Rosenstein can comment).

One more shot from my favorite soapbox, and I'll shut up: Think, and just
about anybody else, will let you use classes in non-apps (DAs, XCMDs, etc).
So far, the MPW linker won't give non-apps globals, hence no method dispatch
tables.

Hope that helps.

=Ned Horvath=

oster@dewey.soe.berkeley.edu (David Phillip Oster) (12/19/89)

In article <257@intek01.UUCP> mark@intek01.UUCP (Mark McWiggins) writes:
_>We're getting discouraged with the speed of MPW C (especially with C++)
_>and are interested in evaluating Think C as a target for C++ 2.0. 
_>But the 32K global data limit is a concern.  Has anyone come up with 
_>a satisfactory workaround?

It is easy if you are writing a C++ to C translator: Just write your
static data into a resource, and generate code to read in the resource at
program initialization time, and reference the globals off of it:

example, a c++ program:

char *banner = "Hello World\n";
main(){
	printf(banner);
}

------------
generates two files: an Rmaker or Rez file with the globals in it
(I'll use RMaker here, because I know it better.)
a.out.rsrc
????????

Type Glob=GNRL
  ,129 (4)
.S
Hello World\0A

------------
and the following .c file:

Ptr __global;
#define __BANNEROFFSET 0
main(){
	Handle __globh;
	if(NULL == (__globalh = GetResource('Glob', 128) )){
		fprintf(stderr, "No Global Resource!\n");
		exit(-1);
	}
	MaxApplZone();
	MoveHHi(__globh);
	HLock(__globh);
	__global = *__globh;

	printf( (char *) (__global+__BANNEROFFSET) );
}

Notes:
We get at globals by doing explicit pointer arithmetic in case THINK C has
restrictions on how big a single struct can be. We initialize __global
once, at the beginning of the program, and after that, it is initialized:
we can just use it as a global base pointer after that.

We call MoveHHi() since the global data handle will be locked for the
entire duration of the program, and we don't want it cluttering the heap.
The call before it make sure it really goes at the _end_ of the
heap. If your translator gives the user control over the amount of program
stack space, you'll want to put a SetApplLimit() before the
MaxApplZone().

The above scheme will work even if you write interrupt level callback
routines in c++, since they will have to set up their own A5, as usual,
and after that, the translated access to globals will work the same as it
does in an application.

--- David Phillip Oster            --  No, I come from Boston. I just work
Arpa: oster@dewey.soe.berkeley.edu --  in cyberspace.
Uucp: {uwvax,decvax}!ucbvax!oster%dewey.soe.berkeley.edu

lsr@Apple.COM (Larry Rosenstein) (12/19/89)

In article <1664@cbnewsk.ATT.COM> ech@cbnewsk.ATT.COM (ned.horvath) writes:

> Object Pascal methods went in the (above A5) jump table at 8 bytes a pop,
> another painful limit with even a moderatedly complex class structure.
> I don't know if that latter situation has been corrected; I know there
> was a lot of concern with it at Apple

The problem has been eased somewhat in MacApp 2.0b9, because we always use 
the optimized method dispatching scheme, even when debugging.

Here's a brief explanation of the problem.

In Object Pascal, the method tables contain the A5 offset of the method's 
jump table entry, since that's the most reliable way to store a pointer to 
a routine (which is what you need in the table).  

In addition, you need to assign an ID to each distinct method.  The way we 
do this is to generate a global selector proc and let the linker assign 
the selector proc a jump table entry.  Then the A5 offset becomes the 
method's ID.

With the optimized method dispatching, there's another step to the 
process.  The linker optimizes methods that are never overridden, making 
them exactly the same as a normal procedure call.  It turns out that 75% 
or so of the method in a typical MacApp program fall into this category.

If a method is never overridden, then it doesn't need a selector proc, and 
it needs a jump table entry only if it is called from another segment.  So 
every optmized method saves 1 and possibly 2 jump table entries.

I think people are looking at more permanent solutions to the problem.

> So far, the MPW linker won't give non-apps globals, hence no method 
dispatch
> tables.

See Tech Note #256.  

Larry Rosenstein, Apple Computer, Inc.
Object Specialist

Internet: lsr@Apple.com   UUCP: {nsc, sun}!apple!lsr
AppleLink: Rosenstein1

ech@cbnewsk.ATT.COM (ned.horvath) (12/20/89)

In article <1664@cbnewsk.ATT.COM> I wrote:
> So far, the MPW linker won't give non-apps globals, hence no method 
> dispatch tables.

From article <5837@internal.Apple.COM>, by lsr@Apple.COM (Larry Rosenstein):
> See Tech Note #256.  

I've seen it.  It's an excellent example of "tell me what you need, I'll
tell you how to live without it."  The suggestion is incomplete -- e.g.
initialized data is not addressed -- and requires the connivance of the
underlying application.  I don't have that connivance with, say, DAs.
Or other drivers.  Or FKEYs. Or any other code resource intended to exist
independently of a PARTICULAR application.  Or even with HyperCard -- the
example in the technote requires careful attention in the Hypercard script
to making sure the XCMD-with-persistent-data is properly set up and torn
down -- an error in that script hoses memory at random, a very user-
unfriendly XCMD to put in the hands of "the rest of us."

I am not satisfied that my needs have been addressed.

=Ned Horvath=

keith@Apple.COM (Keith Rollin) (12/20/89)

In article <1672@cbnewsk.ATT.COM> ech@cbnewsk.ATT.COM (ned.horvath) writes:
>In article <1664@cbnewsk.ATT.COM> I wrote:
>> So far, the MPW linker won't give non-apps globals, hence no method 
>> dispatch tables.
>
>From article <5837@internal.Apple.COM>, by lsr@Apple.COM (Larry Rosenstein):
>> See Tech Note #256.  
>
>I've seen it.  It's an excellent example of "tell me what you need, I'll
>tell you how to live without it."  The suggestion is incomplete -- e.g.
>initialized data is not addressed -- and requires the connivance of the
>underlying application.  I don't have that connivance with, say, DAs.
>Or other drivers.  Or FKEYs. Or any other code resource intended to exist
>independently of a PARTICULAR application.  Or even with HyperCard -- the
>example in the technote requires careful attention in the Hypercard script
>to making sure the XCMD-with-persistent-data is properly set up and torn
>down -- an error in that script hoses memory at random, a very user-
>unfriendly XCMD to put in the hands of "the rest of us."

Ned,

One of us - either you or me - doesn't know what you're talking about. When you
say that it doesn't address the issue of initialized data, what do you mean? After all, MakeA5World is supposed to do this for you. By "this", I mean that it 
creates space for your data, and initializes it all to zero, or with predefined 
values if you are using C or Assembly.

As for DA's, you don't have to rely on any host application. You can just
stick your A5 reference into dCtlStorgage. Same with other drivers. As for
FKEYs, you can use a hack that I used originally, which was to link in a
little assembly PROC that I jammed my A5 reference into.

If you have other issues with the technote (like, for example, I forgot to 
suggest that one call InitGraf when setting up his/her A5 world), then please
let me know. In particular, I'd like to hear your suggestions on getting rid
of the global data without getting some sort of close message. And remember,
you don't HAVE to use persistant global data. Nothing in the rules about that.
I just mentioned it and showed how to do it in the (inevitable) case that
someone asked.

Finally, I'd like to hear where you read "and I'll tell you how to live
without it." I tried to be accomodating. In cases where I said you couldn't do
something, I also tried to explain why. If I failed in some places please let
me know.

All of this applies to EVERY TECHNOTE, and to EVERY READER of this network: if
you have issues with out Technotes...then let us know! Don't go flaming on
the nets. We're here. We listen. These technotes are for you, and we want them
to be useful...

-- 
------------------------------------------------------------------------------
Keith Rollin  ---  Apple Computer, Inc.  ---  Developer Technical Support
INTERNET: keith@apple.com
    UUCP: {decwrl, hoptoad, nsc, sun, amdahl}!apple!keith
"Argue for your Apple, and sure enough, it's yours" - Keith Rollin, Contusions

beard@ux1.lbl.gov (Patrick C Beard) (12/20/89)

In article <37420@apple.Apple.COM> keith@Apple.COM (Keith Rollin) writes:
>Ned,
>
>One of us - either you or me - doesn't know what you're talking about. When you
>say that it doesn't address the issue of initialized data, what do you mean? 
>After all, MakeA5World is supposed to do this for you. By "this", I mean that 
>it creates space for your data, and initializes it all to zero, or with 
>predefined values if you are using C or Assembly.

This is true.  I've taken the tech note to task by creating independent code
resources written in C++ that have global variables with initialized data.
The ability to stuff objects in a code resource is really handy, I've written
simple interfaces (objects) for encapsulating the nastiness that goes along
with this MakeA5World stuff and hides it from me.

>If you have other issues with the technote (like, for example, I forgot to 
>suggest that one call InitGraf when setting up his/her A5 world), then please
>let me know.

I've tried calling InitGraf to initialize my code resource's A5 world.  It
may have been that something else was making it crash, but is this a valid
thing to do so that the A5 world will have properly initialized quickdraw
variables?  I've been stuffing values into the qd globals and it works.

>Finally, I'd like to hear where you read "and I'll tell you how to live
>without it." I tried to be accomodating. In cases where I said you couldn't do
>something, I also tried to explain why. If I failed in some places please let
>me know.

What I want to know is when you will tell us how to create jump tables for
our multi-segmented code resources.  THINK C lets us do that.  Take us one
step closer to the capabilities they provide and the MPW languages will
become a viable development environment for non-application code.

Why?  I want virtual functions in my code resources!  Why must they be
implemented (for C++ or Object Pascal) as jump table entries?  I don't see
why the method tables can't be resolved at data initialization time.  It's
easy to write code that computes the absolute addresses of methods and
stuffs it into a table of function pointers.

Thanks for global data, now give me jump tables!  Then, rewrite MacApp
in C++, then..., well I can dream.


-------------------------------------------------------------------------------
-  Patrick Beard, Macintosh Programmer                        (beard@lbl.gov) -
-  Berkeley Systems, Inc.  ".......<dead air>.......Good day!" - Paul Harvey  -
-------------------------------------------------------------------------------

amanda@mermaid.intercon.com (Amanda Walker) (12/21/89)

In article <37420@apple.Apple.COM>, keith@Apple.COM (Keith Rollin) writes:
> Ned, One of us - either you or me - doesn't know what you're talking about.

Well, Keith, here's a vote in favor of TN#256.  Good work!  It explained
things quite well, I thought...

Amanda Walker
InterCon Systems Corporation
--