wayne@helix.nih.gov (Wayne Rasband) (11/20/90)
Apparently someone at Apple has found one or more bugs in the Memory
Manager in the 32-bit clean ROMs found in the ci,fx, LC, and si that
explains the unbelievably slow NewPtr times I found with NIH Image on the
ci and fx. In one case, a NewPtr call on the ci took an unbelievable 10
seconds, as compared to .25 seconds on the cx!! The bug also seems to
effect NewHandle, since NewHandle calls can be up to 8 times slower on the
ci. The following messages from CompuServe describe the problem, as well
as the Init Apple has developed to fix the problem by patching the Memory
Manager. Sorry, but I don't know where to get the Init.
--wayne rasband
#: 31491 S7/Inside Macintosh 13-Nov-90 08:07:26
Sb: #31478-#ci performance problems
Fm: Ed Ludwig 71210,1545
To: Wayne Rasband 76067,3454
Wayne,
INFO ALERT **** INFO ALERT **** INFO ALERT **** INFO ALERT
A little new information on this front. Evidently, someone from Apple
itself (!!!) has tracked this performance hit down. There was a very
lengthy message on Applelink:Developers Ask Each Other:General Discussion
from a DTS engineer (below) that gave the answer. There is a BUG in the
32-Bit clean memory managerthat basically causes it to go through an
exhaustive search of the heap to get a free block even though it doesn't
have to. This bug is going to require replacing the entire Memory Manager
(about 9K) because there is no way to patch it. I think apples looking
into it. Here is the text of the message:
Mark Szpakowski (TGS Systems, AppleLink CDA0242) asks: --- Why is the IIci
so much slower than the IIx or cx in allocating handles? Like 8 times
slower! We're trying to track down a performance problem using Prograph
on a IIci (every once in a while a big pause before a window is drawn, or
a menu item is executed). I found similar behavior on a LC, and suspect
the same would be true on an SI. Any comments or experience in this
matter? ---
Dick Startz (TSP.STARTZ) responds:
---
I've had exactly the same situation with the IIci using NewPtr rather than
handles. I've documented it and asked DTS for advice.
---
Well, I think that this one has finally been nailed down. When dealing
with Memory Management on the Macintosh, there is a lot of problem dealing
with cause and effect. With as many changes as there were in the ROMs on
the IIci (and IIfx, IIsi, and LC), it's not easy to determine if the
problem is with the Memory Manager or elsewhere (for instance, with some
toolbox routine fragmenting the heap).
After Dick sent in his request to DTS, I spent a few late hours at work
trying to track it down. By carefully controlling the environment the
application ran under, it became apparent that the problem was definitely
a bug in the Memory Manager on those four machines. Those ROMs are
referred to as the "32-bit clean" ROMs, and are identified with the ROM
version $67C. (Dick, by now, you should have your response from DTS
describing the problem in detail).
The fix for this is not going to be simple. The bug is at the very core of
the Memory Manager. Because the Memory Manager is not vectored aside from
its entry points, and because it is self contained, there are no places
for a patch to be hooked in. It is possible that if a fix is put in place,
it will have to be done by replacing the entire Memory Manager (about 9K).
Characteristics of the problem:
-------------------------------
In many of the Memory Manager's operations, it needs to find blocks of
free space. Rather than perform an exhaustive search on the entire heap
every time, the Memory Manager maintains a hint that it stores in the zone
header (AllocPtr). When looking for free space, the Memory Manager starts
at the block pointed to by AllocPtr, proceeding to the end of the heap. If
it doesn't find a block of free space large enough, it starts over from
the beginning of the heap.
The problem is that AllocPtr sometimes gets set with Master Pointer bits
in the high byte. When the comparison is made to see if the pointer is
past the end of the heap, the result always indicates that we HAVE past
the end of the heap, and the search is restarted from the beginning.
Hence, we get a performance degradation.
You may be able to structure your program to avoid this problem as much as
possible. If you can arrange your heap such that relatively few objects
are locked low in memory, or if you can limit your use of non-relocatable
blocks (which are placed low in memory), you can limit the effects of the
bug. By reducing the number of blocks low in the heap, you reduce the
speed impact caused by starting a free block search at the beginning of
the heap.
On the other hand, this approach is realistically impractical... Perhaps a
solution involving sub-zones in your heap would be easier to implement...
- Keith Rollin - Apple Developer Technical Support
DSBB: Devs Ask Each Other: General Discussion 11-12-90
------------------------------------------
We got an answer back from MacDTS on the handle allocation problem with
the IIci. It turns out to be a problem with all machines (IIci, IIfx,
IIsi, LC) that use the new ROMs (ROM version ID $67C) when running in 24
bit mode. Apple originally thought this was due to extra overhead brought
in by using two memory managers (24-bit and 32-bit clean) on these
machines, but after some testing last week it was found that there's a bug
at the heart of the memory manager (in its use of "AllocPtr", which points
at where to start in the search for free blocks). At first glance it
looks like this routine cannot be patched; the entire memory manager
(about 9K) may have to be replaced. I'll keep y'all posted...
#: 31703 S7/Inside Macintosh 15-Nov-90 10:01:11
Sb: #31600-ci performance problems
Fm: Mark Szpakowski 73300,3460
To: John Baxter 71735,1626
Well, the memory manager problems with the new ROM machines (ci, fx, si,
lc) seem to have been zapped, and dramatically at that! Jim Laskey of TGS
Systems, who uses a ci, says "I have a whole new machine". This fixes the
problems with both pointer and handle allocation, as well as the
performance hits that used to plague Prograph on the ci (and the other new
ROM machines), as well as other programs. The unsung hero of all this is
Keith Rollins of Apple, who has been working night and day (literally!) on
this. The first Mem Mngr init fixed the pointer problem, but not the one
with handles. It turned out there were a few more bugs; the second Mem
Mngr init does the job.
-- Mark, TGS Systems
#: 31793 S7/Inside Macintosh 16-Nov-90 14:43:53
Sb: Mem Mngr & ci et al.
Fm: Mark Szpakowski 73300,3460
To: All
Below are our benchmark results for the second version of MMInit, the Init
which fixes Memory Manager problems when running in 24-bit mode on the ci,
fx, si and lc machines. As you can see it works great! I'd be curious
what kind of results the ci gets with a cache card, if anyone out there
has that one.
/* 500k partition */ #define TIMES 10000 #define TICKS (*(long *)0x16a)
main()
{
register long time, loop; /* time in D7, loop in D6 */
register long handle; /* handle in D5 */
MaxApplZone(); /* maximize heap space */
time = TICKS; /* record time */
for( loop = 0; loop < TIMES; loop++ ) /* allocate TIMES handles */
handle = NewHandle( 1 );
time = TICKS - time; /* calculate time difference */
Debugger(); /* view time */ }
/* ci cx ci/cx MMINIT ci ci/MMINIT ci*/
/* totals 1797 222 8:1 192 9.4:1 */Garance_Drosehn@mts.rpi.edu (Garance Drosehn) (11/20/90)
In article <649@nih-csl.nih.gov> wayne@helix.nih.gov (Wayne Rasband) writes: ...(majority of article deleted)... > > / ci cx ci/cx MMINIT ci ci/MMINIT ci*/ > /* totals 1797 222 8:1 192 9.4:1 */ With differences like that, I imagine this MMINIT (wherever it is) will be very popular! Garance_Drosehn@mts.rpi.edu ITS Systems Programmer Rensselaer Polytechnic Institute; Troy, NY. USA
keith@Apple.COM (Keith Rollin) (11/20/90)
In article <9_|^6Z_@rpi.edu> Garance_Drosehn@mts.rpi.edu (Garance Drosehn) writes: >In article <649@nih-csl.nih.gov> > wayne@helix.nih.gov (Wayne Rasband) writes: > ...(majority of article deleted)... >> >> / ci cx ci/cx MMINIT ci ci/MMINIT ci*/ >> /* totals 1797 222 8:1 192 9.4:1 */ > >With differences like that, I imagine this MMINIT (wherever it is) will be >very popular! True, but the Memory Manager bug affects only certain applications. Most applications aren't affected. As near as I can tell, applications that either allocate a large number of non-relocatable blocks of memory with NewPtr or that don't call MoreMasters enough when dealing with thousands of handles (and hence make implicit NewPtr calls in the middle of their programs) are affected. Almost no other applications are impacted (for example, my 26 minute compile of an application under MPW, which is a very Memory Manager intensive program, still takes 26 minutes). -- ------------------------------------------------------------------------------ Keith Rollin --- Apple Computer, Inc. --- Developer Technical Support INTERNET: keith@apple.com UUCP: {decwrl, hoptoad, nsc, sun, amdahl}!apple!keith "Argue for your Apple, and sure enough, it's yours" - Keith Rollin, Contusions
gt0657c@prism.gatech.EDU (geoff george) (11/20/90)
The problem about high bits set in AllocPtr sounds like some Apple programmers forgot to call _StripAddress. How delightful! They must have read TechNote 213 :-) geoff -- geoff george geoff@remvax.gatech.edu (my vax) or gt0657c@prism.gatech.edu (a touch of warmth from GaTech OCS) "Ordinary f---ing people - I hate 'em. Ordinary person spends his life avoiding tense situations; repo man spends his life getting INTO tense situations."
emmayche@dhw68k.cts.com (Mark Hartman) (11/21/90)
In article <9_|^6Z_@rpi.edu> Garance_Drosehn@mts.rpi.edu (Garance Drosehn) writes: >With differences like that, I imagine this MMINIT (wherever it is) will be >very popular! OK, I'll bite - where do we get it? -- ========================================================================= Mark Hartman, N6BMO |"What are you standing uucp: ...{spsd,zardoz,felix}!dhw68k!emmayche |there for? Where do Internet: emmayche@dhw68k.cts.com |you think you are,
d88-jwa@dront.nada.kth.se (Jon W{tte) (11/24/90)
In article <17354@hydra.gatech.EDU> gt0657c@prism.gatech.EDU (geoff george) writes: >The problem about high bits set in AllocPtr sounds like some Apple >programmers forgot to call _StripAddress. How delightful! Pardon an ignorant question: Why replace the memory manager, when a patch that StripAddresses the AllocPtr (or whatever) when you call _NewPtr or _NewHandle ? Or does the problem occur within the manager and the manager doesn't use traps ? h+ h+@nada.kth.se "Moof!(tm)"
keith@Apple.COM (Keith Rollin) (11/24/90)
In article <1990Nov21.014928.19044@dhw68k.cts.com> emmayche@dhw68k.cts.com (Mark Hartman) writes: >In article <9_|^6Z_@rpi.edu> Garance_Drosehn@mts.rpi.edu (Garance Drosehn) >writes: >>With differences like that, I imagine this MMINIT (wherever it is) will be >>very popular! > >OK, I'll bite - where do we get it? The INIT is not stable enough for release. With something as all-pervasive as the Memory Manager, you have to make double-damn sure that there are no crasher bugs. People in-house who have been trying it out report that they have a higher number of random crashes and hangs in normal everyday work. Until those are ironed out, the INIT would be decidedly unpopular. -- ------------------------------------------------------------------------------ Keith Rollin --- Apple Computer, Inc. --- Developer Technical Support INTERNET: keith@apple.com UUCP: {decwrl, hoptoad, nsc, sun, amdahl}!apple!keith "Argue for your Apple, and sure enough, it's yours" - Keith Rollin, Contusions
keith@Apple.COM (Keith Rollin) (11/24/90)
In article <1990Nov23.192937.8359@nada.kth.se> d88-jwa@dront.nada.kth.se (Jon W{tte) writes: >In article <17354@hydra.gatech.EDU> gt0657c@prism.gatech.EDU (geoff george) writes: > >>The problem about high bits set in AllocPtr sounds like some Apple >>programmers forgot to call _StripAddress. How delightful! > >Pardon an ignorant question: > >Why replace the memory manager, when a patch that StripAddresses >the AllocPtr (or whatever) when you call _NewPtr or _NewHandle ? >Or does the problem occur within the manager and the manager >doesn't use traps ? The problem really only shows up in non-relocatable block allocation (i.e. on NewPtr calls). So imagine this: you make a NewPtr call. In order to put the block as low in memory as possible, the Memory Manager must move all relocatable blocks out of the way. OK, so it moves the first one. When it frees up where the block used to be, it sets AllocPtr with the dirty bits. At this time, the hint is bad, and any subsequent searches for free space will require a full heap search. If you must move several relocatable blocks, they all suffer from a full heap search. Since this occurs in a tight, self-contained loop in the Memory Manager, there is no way to sneak in and clear the naughty bits. -- ------------------------------------------------------------------------------ Keith Rollin --- Apple Computer, Inc. --- Developer Technical Support INTERNET: keith@apple.com UUCP: {decwrl, hoptoad, nsc, sun, amdahl}!apple!keith "Argue for your Apple, and sure enough, it's yours" - Keith Rollin, Contusions