[comp.sys.atari.st.tech] Problem with my C program a few general hints about malloc

dmb@wam.umd.edu (David M. Baggett) (04/22/91)

In article <1991Apr21.210056.5685@ccu.umanitoba.ca> umochock@ccu.umanitoba.ca (Russell Ochocki) writes:
>
>I try to Malloc a buffer equal to the size of the file being copyied.  Then I
>do a Fread from the source file, prompt the user to swap disks, fopen the
>destination file, and Fwrite the contents of the buffer.  And, fclose both
>files.  Note: I needed to use Malloc, Fread, and Fwrite since I required long
>ints (not the short expected by malloc, fread, and fwrite).

Although this isn't what you were asking about, I think it's worth
mentioning for the benefit of others who are just starting out:

If you want to allocate more memory than can be specified in an unsigned int,
use calloc.  It takes two parameters which are multiplied together to get
the size of the requested block.  To allocate 128K:

	p = calloc(128, 1024);

As a bonus the memory you get is also cleared.  You could write your own
replacement for Malloc as follows:

char *
my_Malloc(size)
	register long	size;
{
	register unsigned	hi, lo;

	hi = (unsigned) (size >> 16);
	lo = (unsigned) (size & 0x0000FFFF);

	return calloc(hi, lo);
}

(This assumes that a long is 32 bits and an unsigned int is 16, which is
not portable, so it's best to use calloc directly.)

In general it is VERY WISE to avoid using the bios Malloc and Free and
use the malloc, calloc, and free provided with your compiler of
choice.  The bios Malloc has bugs (some still remain for historical
reasons) and is quite ideosyncratic.  Sozobon's malloc and calloc (in
dlibs.a) work quite well, although one caveat to Sozobon users is that
the malloc routines can only manage 1 Meg of memory.  A trivial change
to the dlibs source file malloc.c fixes this, fortunately.

Also, assuming you were using fread and fwrite, you would have to
be sure to specify "b" in the flags ("b" for "binary") to prevent
automatic conversion of CR to CR/LF.

Dave Baggett
dmb%wam.umd.edu@uunet.uu.net

steve@thelake.mn.org (Steve Yelvington) (04/22/91)

[In article <1991Apr22.014758.3846@wam.umd.edu>,
     dmb@wam.umd.edu (David M. Baggett) writes ... ]

> Although this isn't what you were asking about, I think it's worth
> mentioning for the benefit of others who are just starting out:
> 
> If you want to allocate more memory than can be specified in an unsigned int,
> use calloc.  It takes two parameters which are multiplied together to get
> the size of the requested block.  To allocate 128K:
> 
> 	p = calloc(128, 1024);

You also can use lalloc, which works like the library malloc()
function except that it requires a long argument. (Be sure to include
malloc.h when calling any dynamic memory management functions.)

Also, good advice is worth repeating, so...

> In general it is VERY WISE to avoid using the bios Malloc and Free and
> use the malloc, calloc, and free provided with your compiler of
> choice.

(Possible exception: Desk Accessories.)

Back to the original topic of this thread:

(1) Never remove a disk while a file is open.

(2) Never write a program that tempts a user to do (1).

I'm not quite sure why the original program was written, since the
Desktop handles a virtual B drive nicely (just install it!), but if I
were writing such a program:

(1) I'd be sure to close the files before prompting the user to
remove a disk.

(2) I'd call Mediach() after the swap to force an update of the
GEMdos i/o buffers and make sure the action had been performed
properly. 

I might also read the directory to be doubly sure. Beware of Murphy.

 ----
 Steve Yelvington, Marine on St. Croix, Minnesota, USA / steve@thelake.mn.org

optimiza@utrcu1.UUCP (Streng) (04/23/91)

In article <1991Apr22.014758.3846@wam.umd.edu> dmb@wam.umd.edu (David M. Baggett) writes:
>If you want to allocate more memory than can be specified in an unsigned int,
>use calloc.  It takes two parameters which are multiplied together to get
>the size of the requested block.  To allocate 128K:
>	p = calloc(128, 1024);
>As a bonus the memory you get is also cleared.  You could write your own
>replacement for Malloc as follows:
>char *
>my_Malloc(size)
>	register long	size;
>{
>	register unsigned	hi, lo;
>	hi = (unsigned) (size >> 16);
>	lo = (unsigned) (size & 0x0000FFFF);
>	return calloc(hi, lo);
>}
Unfortunately, this will not work the way you intend. Assume you want to
Malloc() a block of 65536 bytes. This leads to hi being 1, and lo being
0. This will calloc() a block of 1*0=0 bytes. What you want to do is
factorize size in two factors that are both small enough to be handled
by calloc. I cannot come up right now with an algorithm that does this
both time and memory efficient.
But, there is another possibility. If you are using an ANSI-C compiler,
the standard says that malloc takes an argument of type size_t, but not
that it will be 16 bit. Turbo C, for example, takes a (maybe even
unsigned) long. (unfortunately, this is not the case with MSDOS-TC,
making ports somewhat more difficult)
---
Henk de Leeuw
optimiza@utwente.nl
optimiza%utrcu1.uucp@uunet.uu.net

dmb@wam.umd.edu (David M. Baggett) (04/23/91)

In article <827@utrcu1.UUCP> optimiza@utrcu1.UUCP (De Leeuw) writes:
>In article <1991Apr22.014758.3846@wam.umd.edu> dmb@wam.umd.edu (David M. Baggett) writes:
>>
>> [my brain-damaged code]
>>
>Unfortunately, this will not work the way you intend. Assume you want to
>Malloc() a block of 65536 bytes. This leads to hi being 1, and lo being
>0. This will calloc() a block of 1*0=0 bytes. What you want to do is
>factorize size in two factors that are both small enough to be handled
>by calloc. I cannot come up right now with an algorithm that does this
>both time and memory efficient.

Oops, looks like someone broke into my account and posted something stupid!
I'll make sure I use better passwords in the future.  :-)

Actually what I usually do in practice is something like

	calloc((unsigned) (bigsize / 32), 32);

which will work for sizes up to 32*65536 bytes.  You can always increase
the "chunking factor" here at the expense of a few wasted bytes allocated.
This is basically what you describe but "by hand."

Another caveat re: Malloc is that it takes a _long_ parameter:

	Malloc(32L);

instead of

	Malloc(32);	/* <- This will cause bad things to happen */

This type of bug is particularly difficult to track down.  It would be
nice if the os-binding for Malloc had a (long) typecast in it to solve
problems like these -- oh well.

Dave Baggett
dmb%wam.umd.edu@uunet.uu.net

apratt@atari.UUCP (Allan Pratt) (04/26/91)

steve@thelake.mn.org (Steve Yelvington) writes:

>[In article <1991Apr22.014758.3846@wam.umd.edu>,
>     dmb@wam.umd.edu (David M. Baggett) writes ... ]

>> If you want to allocate more memory than can be specified in an unsigned int,
>> use calloc.  It takes two parameters which are multiplied together to get
>> the size of the requested block.  To allocate 128K:
>> 
>> 	p = calloc(128, 1024);

In general, this is not a fix.  I have seen allocators which call malloc
from calloc after multiplying the two args together AS INTS, not longs. So
this does you no good at all.  Some compilers work and some don't in this
respect.  You pays your money and you takes your chances.

>You also can use lalloc, which works like the library malloc()
>function except that it requires a long argument. (Be sure to include
>malloc.h when calling any dynamic memory management functions.)

Much better solution, if your library has lalloc. Some have lmalloc, so try
both.  (I forget if the original poster said what compiler/library he was
using.)

>> In general it is VERY WISE to avoid using the bios Malloc and Free and
>> use the malloc, calloc, and free provided with your compiler of
>> choice.

Humph.  TOS 1.4 and upwards have NO PROBLEMS with Malloc that have been
reported to me.

============================================
Opinions expressed above do not necessarily	-- Allan Pratt, Atari Corp.
reflect those of Atari Corp. or anyone else.	  ...ames!atari!apratt

dmb@wam.umd.edu (David M. Baggett) (04/26/91)

In refrence to my suggestion to avoid Bios Malloc, Allan Pratt writes:
>
>Humph.  TOS 1.4 and upwards have NO PROBLEMS with Malloc that have been
>reported to me.

This has little bearing on the fact that TOS 1.2 and downwards DO have
problems.  I want my software to run on all ST's.  Please understand
that I wasn't trying to criticize you; in fact I realize that the whole
situation has been made worse for you because lots of programmmers have
_relied_ on the earlier bugs.

Using Unix-compatible malloc routines provided in a compiler's libc.a
is better than using Atari's Malloc binding because:

	1) It works on all ST's (assuming said malloc is properly written)
	2) It's more portable (does not assume longs are bigger than ints)
	3) It's not prone to Malloc(1) versus Malloc(1L) bugs

On the other hand, the drawbacks are:

	1) It's slower
	2) It adds code size to your executable

It would be great if I could assume all ST's out there were running TOS
1.4, but I can't.  I should think you'd want us to be writing stuff
that is maximally machine-indpendent.  I'm not being snide here; this
is a thorny issue that ST programmers have to deal with.  I've written
lots of code using Malloc and lots of code using malloc -- I've never
had problems with the Dlibs malloc routine, while the Bios Malloc
(under TOS 1.2, I admit) caused me tremendous grief.

If you thought I was just making some off the cuff slanderous remark,
you were mistaken.  I posted the advice I gave based on 4 years of
heavy-duty ST programming, simply hoping to prevent others from wasting
their time dealing with bugs and ideosyncracies I learned about the
hard way several years ago.

If Atari wants my opinion (which they probably don't), I think they
should release "official" source code for malloc, calloc, and lalloc
that works on all ST's and coalesces memory correctly.  (Perhaps this
would check the TOS version and just call the Bios for 1.4 and up.)
Then we'd have a common denominator.  This would also solve the problem
of how to allocate more than 64K at a time.

Dave Baggett
dmb%wam.umd.edu@uunet.uu.net

bjoern@drdhh.hanse.de (Bjoern Kriews) (04/26/91)

From dmb@wam.umd.edu (David M. Baggett):
> Oops, looks like someone broke into my account and posted something stupid!

Oops, looks like someone did it once again .-)

> 	calloc((unsigned) (bigsize / 32), 32);

Just think about bigsize == 320002...

        ,ppp                     .ppp
    pp pp``Tp                pp pp```pp
  __   __    BE__          ,_   __    6B__
  VV ,pVV  .ppJBpppp       "V ,pVV   pppBLppp
     ``    |BBBBBBBL          ``     BBBBBBBF
       j[BBBBBBBBBBBBBFj        jjBBBBBBBBBBBBBEj
       BBBWWBBBBBBBBBBBB        BBBWWBBBBBBBBBBBBL
     /pBBBppBBBBBBBBBBBBpp    ,pBBBppBBBBBBBBBBBBpp
     6BBBBBBBBBBBB`}BBBBBE    [BBBBBBBBBBBB`TBBBBBB
     ``BBBBBBBBBBBj}BBBB``     `BBBBBBBBBBBj|BBBBL`
       VVBBBBBBBp_BBBBRV        VVBBBBBBBL_BBBBBVV
        '"BBBBBBBBBBB""          '"BBBBBBBBBBB""
          ```BBBBB```              ```6BBBB```
                 (not necessarily, but...)
                 
Greetings, Bjoern


---
bjoern@drdhh.hanse.de = Bjoern Kriews / Stormsweg 6 / 2000 Hamburg 76 / FRG
"gaaga mahwe Bjoern urgl ufzae Turbo-C bnub"     (J. Willamowius)

dmb@wam.umd.edu (David M. Baggett) (04/30/91)

In article <2941.04.91@drdhh.hanse.de> bjoern@drdhh.hanse.de (Bjoern Kriews) writes:
>From dmb@wam.umd.edu (David M. Baggett):
>> Oops, looks like someone broke into my account and posted something stupid!
>
>Oops, looks like someone did it once again .-)
>
>> 	calloc((unsigned) (bigsize / 32), 32);
>
>Just think about bigsize == 320002...

Would you believe it?  TWO forged messages in a row!  The audacity of it!
And right after I changed my password to something no one would guess:
"password".  (Oops!  Now everyone knows!)

What the loser who posted that MEANT was:

#define CHUNK 32	/* or 64, 128, or whatever */

	calloc((unsigned) (bigsize / CHUNK + 1), CHUNK);

You see here what the intruder was feebly trying to get at about "space
wastage" -- you get a few extra unused bytes at the end.  Of course,
that's better than writing a few extra bytes _past_ the end, which is
what I think you were trying to get at with your picture.  :-)

I hope whoever broke into my account saw your reply!

BIFF

uh, I mean

Dave Baggett
dmb%wam.umd.edu@uunet.uu.net

bjoern@drdhh.hanse.de (Bjoern Kriews) (05/01/91)

From article <2913@atari.UUCP>, by apratt@atari.UUCP (Allan Pratt):
> Humph.  TOS 1.4 and upwards have NO PROBLEMS with Malloc that have been
> reported to me.

Hmm, try Malloc'ing little blocks until the OS returns NULL because there
is no internal memory left for the MD's.
Then Fopen() something.
I think Fopen should return -39, but it will blow up the 'OUT OF' message.

There are still other problems:
Fdup'ing an already Fdup'ed handle causes the 'OUT OF' too.

I just mailed you (Allan) a better description including sample source.
If you don't get it via e-mail, please let me know.

Greetings, Bjoern

---
bjoern@drdhh.hanse.de = Bjoern Kriews / Stormsweg 6 / 2000 Hamburg 76 / FRG
"gaaga mahwe Bjoern urgl ufzae Turbo-C bnub"     (J. Willamowius)