[comp.sys.mac.programmer] Need large array on Think C.

stoms@castor.ncgia.ucsb.edu (David Stoms) (05/26/90)

In article <265986A9.26645@paris.ics.uci.edu> jchoi@paris.ics.uci.edu (John Choi) writes:
>     Sorry for this trival question, but I really need to get this
>done.  When I declare an array ('char arr[100][300]'), the complier
>gives me an 'illegal array bounds' error.  Is there any way I can
>increase this limit?

I'm getting tired of this question so I think I'll try to give a generic
answer to cover any further problems.

When you want lots of variable space >32K in Think C or any compiler
that limits globals to 32K, you have to use the Memory Manager (gasp!).
If you want a big array, such as "char arr[100][400]" then you need to
sever your dependancy from the compiler and get your hands dirty.

To allocate this array:

char	*arr;

arr = NewPtr((Size)100*400);

Then to use the array you can index like this:

x = 20; y = 32;
*(arr+x+y*100) = 'H';

Thats it! If you want to use a handle the only big difference is:

*(*arr+x+y*100)  << note the extra * >>

If you do this, be sure to lock down the handle when your r.h.s. could
move memory.

Josh.

dudek@ai.toronto.edu (Gregory Dudek) (05/28/90)

In article <5506@hub.ucsb.edu> stoms@castor.ncgia.ucsb.edu () writes:
>In article <265986A9.26645@paris.ics.uci.edu> jchoi@paris.ics.uci.edu (John Choi) writes:
>>     Sorry for this trival question, but I really need to get this
>>done.  When I declare an array ('char arr[100][300]'), the complier
>>gives me an 'illegal array bounds' error.  Is there any way I can
>>increase this limit?
>
>
>When you want lots of variable space >32K in Think C or any compiler
>that limits globals to 32K, you have to use the Memory Manager (gasp!).
>If you want a big array, such as "char arr[100][400]" then you need to
>sever your dependancy from the compiler and get your hands dirty.
>
>To allocate this array:
>
>char	*arr;
>
>arr = NewPtr((Size)100*400);
>
>Then to use the array you can index like this:
>x = 20; y = 32;
>*(arr+x+y*100) = 'H';
>
>Thats it! If you want to use a handle the only big difference is:
>*(*arr+x+y*100)  << note the extra * >>
>
>If you do this, be sure to lock down the handle when your r.h.s. could

  Although the method above works fine, I find it a bit ugly.  Not only
does sticking in calls to NewPtr explicitly lose portability, but the
indexing scheme is much less readable than regular arrays.
  Finally, you may want to have the arrays initialized to zero, like ``real''
ones.

  I prefer the following scheme which is much more isomorphic to
the commonly used sytax.
  Note, also, that since the arrays are allocated using pointers not
handles, no locking down of memory is required at any time.


1) Add this header code:
=======================

#define DCLARRAY(type,name,y,x)	type *name[y]
#define INITARRAY(type,name,y,x)	allocate(name,(int)sizeof(type),y,x)


allocate(array,elementsize,y,x)
char *array[];
int elementsize;
int y,x;
{
    register int ix, iy;
    for (iy=0;iy<y;iy++) {
	array[iy] = NewPtr((long)(x*(long)elementsize));
	if (!array[iy]) exit(1);
    }
    /* init array to zero here, if desired */
}


2) Replace array declarations of the form:
  float foo[999][566]       with
  DCLARRAY(float,foo,999,566)

3) Insert this before the array would be used:
	INITARRAY(float,foo,999,566)

4) Use the array like normal, i.e. foo[23][21]
   

   Greg Dudek
-- 
Dept. of Computer Science (vision group)    University of Toronto
e-mail:  
    soon to be: dudek@McRCIM.McGill.EDU (usable now, preferred after June/90)
    for now:    dudek@ai.utoronto.ca (or dudek@ai.toronto.edu)
UUCP: {uunet,decvax,linus,pyramid,

J.COOK@ENS.Prime.COM (05/29/90)

There has been some discussion of a note from stoms@castor.ncgia.ucsb.edu
>>     Sorry for this trival question, but I really need to get this
>>done.  When I declare an array ('char arr[100][300]'), the complier
>>gives me an 'illegal array bounds' error.  Is there any way I can
>>increase this limit?

I find it too bad that all solutions in C involve allocating the array
at runtime and then referencing the variable through a pointer variable.
The resulting loss of efficiency due to indirection is disappointing.
It is particularly bad when you realize that compilers (both C and Pascal)
tend to not perform cross-statement register tracking (I admit I am unsure
about MPW).  The effect is that the pointer to the array must be reloaded
into a register for each statement that references the array, regardless
of whether the pointer is already present in a register or not.

This is a general problem.  Compilers on the Mac lack optimization.
Dataflow analysis?  Flow analysis?  Code migration?  Constant subexpression
optimization?  Loop invarient removal?  Cross-statement register tracking?
Back-patching?  Never seen them on the Mac.  Compiler-writers cite lack of
demand for it.

Sigh.  It would seem to me that good code optimization could noticeably
improve the performance of many applications.  Just by improving the code
efficiency, you could gain much of the effect of some hardware accelerators
for the price of a compilation.

Jim Cook
<J.COOK@ENS.Prime.COM>
"Just my misinformation, not Prime's."

J.COOK@ENS.Prime.COM (05/30/90)

========> forwarded message from eco8941@ecostat.aau.dk

There is not more problems in C than in other languages, you
can get all the spped you want by using register variables.
something like this should work:

typedef char mytype[100][300];

mytype *globalarray;

main()
{
   globalarray = malloc( (unsigned long) sizeof(mytype) );
   /*            ^^^^^^ use your favourite allocation routine here */
   .
   .
}

usearray()
{
   register mytype *myarray;

   myarray[25][50] = 'a';    /* no overhead, direct in register */
}

I haven't checked if I could make globalarray of register storage class,
but that would remove the little overhead introduced in usearray().


========================================================================
| Povl H. Pedersen      |  InterNet address: ECO8941@ecostat.aau.dk    |
| Student               |----------------------------------------------|
| Dept. of Economics    | Spare time Mac Programmer                    |
| Aarhus Universitet    | Own Hardware: Mac SE, 4MB + 49MB SCSI HD     |
| DENMARK               | Interests: Macintosh, C,  boardgames         |
|======================================================================|
[ Why I don't keep things in the right order ?    I understand Chaos ! ]
========================================================================

d88-jwa@nada.kth.se (Jon W{tte) (05/30/90)

In article <1232500001@ENS.Prime.COM>, J.COOK@ENS.Prime.COM writes:
> 
> There has been some discussion of a note from stoms@castor.ncgia.ucsb.edu

> >>done.  When I declare an array ('char arr[100][300]'), the complier
> >>gives me an 'illegal array bounds' error.  Is there any way I can

> I find it too bad that all solutions in C involve allocating the array
> at runtime and then referencing the variable through a pointer variable.
> The resulting loss of efficiency due to indirection is disappointing.

What do you think char foo[100][300] means anyway ? It's an array
of 100 references to 300-char arrays. You could use foo[100] as a
char pointer. So what's the big problem with allocating the block
containing the 100 pointers, and then allocating the 100 blocks in
a loop at the beginning ? You could always move the handle high,
lock it down down, and forget about it's handle nature, meaning that
the same code may be used for this technique (char ** foo) as for the
"ugly" (char foo [N][M]) way.

> It is particularly bad when you realize that compilers (both C and Pascal)
> tend to not perform cross-statement register tracking (I admit I am unsure
> about MPW).  The effect is that the pointer to the array must be reloaded

MPW C actually does some optimization. I like THINK C better, though,
for other reasons. And I'm not afraid of a little asm {} :-)

You could always declare a register char * variable, and assign it
before you wanted to use it multiple times, resulting in a guaranteed
performace increase.

> Dataflow analysis?  Flow analysis?  Code migration?  Constant subexpression
> optimization?  Loop invarient removal?  Cross-statement register tracking?

At least constant subexpression's done...

> Sigh.  It would seem to me that good code optimization could noticeably
> improve the performance of many applications.  Just by improving the code

Want to port gcc ? ;-)


	Jon W{tte, Stockholm, Sweden, h+@nada.kth.se

spencer@eecs.umich.edu (Spencer W. Thomas) (05/31/90)

In article <1232500001@ENS.Prime.COM> J.COOK@ENS.Prime.COM writes:

> I find it too bad that all solutions in C involve allocating the array
> at runtime and then referencing the variable through a pointer variable.
> The resulting loss of efficiency due to indirection is disappointing.

Except for the fact that ALL variable reference on the Mac is indirect
through a register, even global and/or "static" variables.  Instead
of using A5 or A6, you are using one of A2-A4.  So what?  The access
time is probably overwhelmed by the array index calculation, anyway.

--
=Spencer (spencer@eecs.umich.edu)

d88-jwa@nada.kth.se (Jon W{tte) (05/31/90)

In article <1232500002@ENS.Prime.COM>, J.COOK@ENS.Prime.COM writes:
> ========> forwarded message from eco8941@ecostat.aau.dk

> typedef char mytype[100][300];

>    globalarray = malloc( (unsigned long) sizeof(mytype) );
>    /*            ^^^^^^ use your favourite allocation routine here */

>    myarray[25][50] = 'a';    /* no overhead, direct in register */

This most definately won't work, since you NEVER have anything but
one-dimentional arrays in C. You'll have to go through the procedure
of allocating pointers to the various rows, and then allocating the
rows. I posted how to do this one or two days ago.

	Jon W{tte, Stockholm, Sweden, h+@nada.kth.se

bruner@sp15.csrd.uiuc.edu (John Bruner) (06/01/90)

In article <1990May31.103925.13522@kth.se>, d88-jwa@nada (Jon W{tte) writes:
>In article <1232500002@ENS.Prime.COM>, J.COOK@ENS.Prime.COM writes:
>
>> typedef char mytype[100][300];
>>    globalarray = malloc( (unsigned long) sizeof(mytype) );
>>    myarray[25][50] = 'a';    /* no overhead, direct in register */
>
>This most definately won't work, since you NEVER have anything but
>one-dimentional arrays in C. You'll have to go through the procedure
>of allocating pointers to the various rows, and then allocating the
>rows....

It is possible to dynamically allocate an N-dimensional array with a
single call to malloc() if the last N-1 dimensions are known at
compile time.  Using this two-dimensional example, declare

	typedef char mytype[100][300];
	char (*globalarray)[300];

and allocate it with

	globalarray = (char (*)[300])malloc((unsigned long)sizeof(mytype));

or, with typedefs

	typedef char char300[300];
	typedef char300 mytype[100];
	char300 *globalarray;

	globalarray = (char300 *)malloc((unsigned long)sizeof(mytype));

You can then say

	globalarray[25][50] = 'a';

The overhead you incur depends upon the storage class of "globalarray"
(e.g., whether it is register) and the degree to which the compiler
optimizes the code.  However, it requires less memory and fewer levels
of indirection than the alternate approach of setting up a
one-dimensional array of pointers to the rows in the matrix.

You must set up an array of pointers if the second dimension (300 in
this case) is not a compile-time constant.
--
John Bruner	Center for Supercomputing R&D, University of Illinois
	bruner@uicsrd.csrd.uiuc.edu	(217) 244-4476	

zen@dhw68k.cts.com (Gary Grossman) (06/02/90)

There's no reason to go through all that mishmash with an array of
pointers when C views an array the same way it does a pointer.

This is the simplest way I can think of to do this:

typedef char FOO[300];

FOO *buf;

buf = NewPtr((Size) 100*sizeof(FOO));

Then buf could be treated as a normal array, requiring far less space
than allocating an array of pointers.  I think the FOO *buf could
also be expressed as char *(buf[300]) but I am not sure.

Gary