[comp.windows.ms.programmer] Allocating a lot of memory

rob@pcad.UUCP (Ralph Brown) (05/03/91)

Running Win 3.0 in enhanced mode, there seems to be some limit to how
much memory a windows app can allocate which is much less than the
amount of free virtual memory. With over 7Mb free, I could only do
a Global32Alloc of about 1.3 Mb. A second copy of the same program could
simultaneously get about the same amount leaving about 5Mb still free.

I've heard that Windows imposes an arbitrary limit of 1/4 of the available
virtual memory per process. This all seems hard to understand since on a
single user system the problem of a user hogging all the resources isn't
really an issue.

Anyone have any comments or experiance with this?

Ralph

ferdie@coyote.datalog.com (fred jarvis) (05/05/91)

rob@pcad.UUCP (Ralph Brown) writes:

> 
> Running Win 3.0 in enhanced mode, there seems to be some limit to how
> much memory a windows app can allocate which is much less than the
> amount of free virtual memory. With over 7Mb free, I could only do
> a Global32Alloc of about 1.3 Mb. A second copy of the same program could
> simultaneously get about the same amount leaving about 5Mb still free.
    I have found that although there seems to be a limit on the amount
of global memory you can get with one call to GlobalAlloc (about .8 meg
on our 8 meg system), you can make repeated calls to GlobalAlloc to
get more memory (each with a different handle).
    Don't ask me why--I've given up trying to understand this stuff.

    Fred Jarvis  ferdie@coyote.datalog.com

donc@microsoft.UUCP (Don CORBITT) (05/11/91)

>rob@pcad.UUCP (Ralph Brown) writes:
>> Running Win 3.0 in enhanced mode, there seems to be some limit to how
>> much memory a windows app can allocate which is much less than the
>> amount of free virtual memory. With over 7Mb free, I could only do
>> a Global32Alloc of about 1.3 Mb. A second copy of the same program could
>> simultaneously get about the same amount leaving about 5Mb still free.

I'm not sure what's happening on your systems.  It is possible your heap
is filled with pagelocked objects, but real unlikely.  Are you calling
GlobalAlloc(), or are you calling GlobalCompact(0)?  Note that under
Enchanted mode, GlobalCompact() doesn't really compact the heap, it just
tells you the largest block you can get _without_ compacting.  This is
on purpose, since compacting the heap on a page-swapping system takes a
lot of time and disk access.  If you really want a lot of memory, just
allocate it.  This will convince Windows that you are serious, and force
heap compaction.

I'm able to routinely allocate 4 MB (and sometimes 8MB) on my system with
8MB ram, (6MB available to Windows).  You should be able to, also.
--
Don Corbitt, MSWinDev

cgeisler@maytag.waterloo.edu (Craig Eisler) (05/13/91)

rob@pcad.UUCP (Ralph Brown) writes:
> Running Win 3.0 in enhanced mode, there seems to be some limit to how
> much memory a windows app can allocate which is much less than the
> amount of free virtual memory. With over 7Mb free, I could only do
> a Global32Alloc of about 1.3 Mb. A second copy of the same program could
> simultaneously get about the same amount leaving about 5Mb still free.

Windows 3.0 has some insane restriction on your real-to-virtual ratio 
when allocating memory in enhanced mode; on a system with 64MB of virtual
memory, the maximum I have been able to Global32Alloc is about 18MB.

I have been told by those in the know that this silly restriction will
leave us in Windows 3.1, but for now, we're just stuck.

craig
-- 
Craig Eisler, still hiding from the real world.
University of Waterloo, Waterloo, Ontario.

rob@pcad.UUCP (Ralph Brown) (05/14/91)

In article <72271@microsoft.UUCP>, donc@microsoft.UUCP (Don CORBITT) writes:
> 
> I'm not sure what's happening on your systems.  It is possible your heap
> is filled with pagelocked objects, but real unlikely.  Are you calling
> GlobalAlloc(), or are you calling GlobalCompact(0)?  Note that under
> Enchanted mode, GlobalCompact() doesn't really compact the heap, it just
> tells you the largest block you can get _without_ compacting.  This is
> on purpose, since compacting the heap on a page-swapping system takes a
> lot of time and disk access.  If you really want a lot of memory, just
> allocate it.  This will convince Windows that you are serious, and force
> heap compaction.
> 
> I'm able to routinely allocate 4 MB (and sometimes 8MB) on my system with
> 8MB ram, (6MB available to Windows).  You should be able to, also.
> --
> Don Corbitt, MSWinDev


What I`m seeing seems pretty odd. My program just does a GlobalCompact(0),
and a GetFreeSpace to see what Windows will admit to. It uses a starting
memory size and and increment value both of which start at 1 Meg. It then
attempts to GlobalAlloc the memory size amount, and if it succeeds, adds the
increment to the size, frees the previous block and tries again. If it
fails, it goes back to the last size that worked,  cuts the increment in half,
and tries again.

What I see when I run this (confirmed by heapwalker) is the first time I
run, I get 1.3 Meg. I then reset everything (without exiting), and try
again. The second or third time I run, I consistantly get 3.5 Meg and from
then on in the same session, it alternates between 1.3 and 3.5. However, if
I'm in one of the passes where I've gotten over 2 MB, if I reset without
exiting I can consistantly get at least 3 MB every time. If I continue asking
for larger blocks till it fails on a GlobalAlloc, then I have to go through
a cycle of 1.3M.

The results for max memory are:

1.3 1.3 3.5 1.3 3.5 1.3 3 3 3 3 3 3.5 1.3 3.5 ...

If I reduce my swapfile size so there's only 3 MB free when I start, the
same behavior occurs, however it is at lower values of 1.1 MB and 2 MB.

Do you have a large swap file? You said that occasionally you can get 8MB,
normally 4 MB. What distinguishes these two cases?

Ralph

rob@pcad.UUCP (Ralph Brown) (05/15/91)

In article <1991May13.003243.26563@maytag.waterloo.edu>, cgeisler@maytag.waterloo.edu (Craig Eisler) writes:
> Windows 3.0 has some insane restriction on your real-to-virtual ratio 
> when allocating memory in enhanced mode; on a system with 64MB of virtual
> memory, the maximum I have been able to Global32Alloc is about 18MB.
> 
> I have been told by those in the know that this silly restriction will
> leave us in Windows 3.1, but for now, we're just stuck.

Yes, I've heard the same thing, but I think it's more complex than that :-<
I wrote a small program that just attempts to allocate the largest block it
can get by allocing and freeing larger and larger pieces, or attempts to
get a big piece all at once.

In a system starting with 7.7MB from GetFreeMem, and nothing else running
it got 3.1MB using repeatedly larger blocks.

However, if I start the program and just try to get 3 MB in one piece without
first doing the incremental size mallocs, it fails, consistantly,
in fact, I can't get 2 MB in one shot. I then do the allocation in pieces
(same invocation) and I get back to 3.1 MB and then can from then on
alloc the full 3.1 in one shot.

It seems like the heap needs a bit of conditioning when it starts. I don't
see why I can't get a big hunk of the other 4M however.

On a related note, if I alloc 3MB and keep it, then attempt to run
WindowPaint, I get an insufficient memory error despite having over 4MB
and 83% of resources free. This makes me wonder if it is possible to use over
about half of virtual memory at all!

Ralph

donc@microsoft.UUCP (Don CORBITT) (05/21/91)

In article <522@pcad.UUCP> rob@pcad.UUCP (Ralph Brown) writes:
>In article <72271@microsoft.UUCP>, donc@microsoft.UUCP (Don CORBITT) writes:
 [I (donc) make suggestions]
>> I'm able to routinely allocate 4 MB (and sometimes 8MB) on my system with
>> 8MB ram, (6MB available to Windows).  You should be able to, also.
 [He (Ralph) supplies details, which include oscillation 1.x MB to 3.x MB]>
>Do you have a large swap file? You said that occasionally you can get 8MB,
>normally 4 MB. What distinguishes these two cases?
>
>Ralph

This was a fun question, so I looked into it further.  PageLocked (uncommon)
and Fixed (too common!) objects fragment the protect-mode heap.  If you
had ToolHelp.DLL, you could scan the heap in your program, and find which
if any memory blocks are fixed or pagelocked.  This would let us know if
there was a believable explanation for this behaviour.

Answering your question, I ran a quick, 10 line program (output was a 
MessageBox() :-), so I don't know why I got 4MB some times, and 8MB others.
However, the granularity was 2X each request, so it may be that 7.999MB was
available when I say I only got 4MB.
--
Don Corbitt, uunet!microsoft!donc.  Windows Developer.
Mail flames, post apologies.

rob@pcad.UUCP (Ralph Brown) (05/23/91)

In article <72462@microsoft.UUCP>, donc@microsoft.UUCP (Don CORBITT) writes:
> In article <522@pcad.UUCP> rob@pcad.UUCP (Ralph Brown) writes:
> >In article <72271@microsoft.UUCP>, donc@microsoft.UUCP (Don CORBITT) writes:
>  [donc makes suggestions]
> >> I'm able to routinely allocate 4 MB (and sometimes 8MB) on my system with
> >> 8MB ram, (6MB available to Windows).  You should be able to, also.
>  [(Ralph) supplies details, which include oscillation 1.x MB to 3.x MB]>
>  donc makes more suggestions:
> This was a fun question, so I looked into it further.  PageLocked (uncommon)
> and Fixed (too common!) objects fragment the protect-mode heap.  If you
> had ToolHelp.DLL, you could scan the heap in your program, and find which
> if any memory blocks are fixed or pagelocked.  This would let us know if
> there was a believable explanation for this behaviour.
> 
> Answering your question, I ran a quick, 10 line program (output was a 
> MessageBox() :-), so I don't know why I got 4MB some times, and 8MB others.
> However, the granularity was 2X each request, so it may be that 7.999MB was
> available when I say I only got 4MB.

You may have missed a later posting I (ralph) made. It seems that if
you sneak up on a big allocation you can get much more than just asking for it.
I wrote a small (50 line) program that output to the window title to keep
the heap as simple as possible which kept asking for larger chunks, then
freeing them. If it got a refusal, it went back to the last one that
worked and cut the size of the next increase in half. If it succeeds it
just frees the memory, adds the same increment and tries again.

What happens is that the first time it runs in an instance it hits a
temporary barrier where an allocation of 2MB fails, then it cuts the
increment down to 1953 bytes in several steps and from then on increasing
size requests succeed up to about 3.2 MB at which time it binary searches
to about 3.1 MB. If this is then run again in the same instance with the
starting point at 1MB and the starting increment at 1MB (same as the first
time) it gets 3MB, fails at 4MB and settles rapidly on 3.1 MB. It seems like
there is some sort of heap conditioning going on. Could this be related to
GlobalCompact not dealing with virtual memory?

The max size I can get (3.1MB) seems to not be a function of my virtual
memory size. If I increase the swap file a lot (from 4MB to 16 MB with
4MB RAM), there is no increase in the max mem size. This seems to be all I
can allocate in all processes also. If I get the 3.1MB I cannot start anything
else although I have over 10MB and 70% of resources free.

No, I don't have TOOLS.DLL, nor for that matter have I ever heard of it,
did part of my SDK get mangled :->.

Ralph

chrisr@cognos.UUCP (Christine Roine) (05/25/91)

In article <524@pcad.UUCP> rob@pcad.UUCP (Ralph Brown) writes:
>The max size I can get (3.1MB) seems to not be a function of my virtual
>memory size. If I increase the swap file a lot (from 4MB to 16 MB with
>4MB RAM), there is no increase in the max mem size.

Right.  Windows is set up so that Virtual Memory is limited to no more than
4 * Physical Memory.  Increasing your swap file to greater than this size
is useless.


-- 
Christine Roine          Cognos Incorporated     S-mail: P.O. Box 9707
Voice: (613) 738-1338 x6205                              3755 Riverside Drive
  FAX: (613) 738-0002                                    Ottawa, Ontario
 UUCP: mitel!cunews!cognos!chrisr                        CANADA  K1G 3Z4

Christine.Roine@sunbrk.FidoNet.Org (Christine Roine) (05/25/91)

Reply-To: chrisr@cognos.UUCP (Christine Roine)

In article <524@pcad.UUCP> rob@pcad.UUCP (Ralph Brown) writes:
>The max size I can get (3.1MB) seems to not be a function of my virtual
>memory size. If I increase the swap file a lot (from 4MB to 16 MB with
>4MB RAM), there is no increase in the max mem size.

Right.  Windows is set up so that Virtual Memory is limited to no more than
4 * Physical Memory.  Increasing your swap file to greater than this size
is useless.


-- 
Christine Roine          Cognos Incorporated     S-mail: P.O. Box 9707
Voice: (613) 738-1338 x6205                              3755 Riverside Drive
  FAX: (613) 738-0002                                    Ottawa, Ontario
 UUCP: mitel!cunews!cognos!chrisr                        CANADA  K1G 3Z4

 * Origin: Seaeast - Fidonet<->Usenet Gateway - sunbrk (1:343/15.0)