[comp.sys.mac.programmer] A Tale of Two Bugs

lippin@jell-o.berkeley.edu (The Apathist) (05/15/88)

	I've been working on a custom MDEF, changing the InvertRect
calls to paint & redraw so it will work on color monitors.  I thought
it was working, but I kept having an uneasy feeling when I finished
selecting an item with it.  After watching it closely a few times, I
figured out why: the menu wasn't blinking when I let up the mouse.

	With some difficulty, I tracked down this problem -- part of
it is Apple's fault, and part is the fault of Lightspeed C.

	Apple's problem is that to make an item blink, MenuSelect
calls the MDEF with whichitem pointng to the low-memory global
ToolScratch, which alternately contains zero and the number of the
item selected.  From my understanding of the rules for ToolScratch,
this is bound to lose; any toolbox call that the MDEF makes may stomp
all over it.  But I guess some stingy Apple programmer wanted to save
a couple of bytes of stack space.

	However, I seem to have been lucky.  Nothing I called was
messing up ToolScratch.

	Here's where LSC comes in.  It's using ToolScratch too, and
dumps the address of the MDEF into it during its code resource
initialization code.  And thus making the same mistake Apple did: that
address could easily be trashed before it's used.  I don't know of any
truly safe place LSC could have used instead, but if there isn't, I'll
let them have one: just take the last four bytes of ApplScratch.  If
more space is needed, put a handle there.  They can save/restore it to
avoid reentrancy problems, and it will break few existing sources.  It
need not break *any* future sources if it was *documented*, like the
use of ToolScratch should have been.

	These problems are in LSC 2.15 and the 5.0 system software.
What versions will they *not* be in?

					--Tom Lippincott
					..ucbvax!math!lippin
					  lippin@math.berkeley.edu

	"Well, you can't eat that raw!"
					--Mrs. Premise

Michael_mkahl_Kahl@cup.portal.com (06/26/88)

LSC's use of ToolScratch is a holdover from the days before it had
an inline assembler.  The address of the code resource can now be
accessed directly from register A0.  I have removed the reference
to ToolScratch in version 3.0 (shipping imminently).

That said, however, I would like to defend the use of ToolScratch
by earlier versions of LSC.  It is perfectly legitimate!  There are
two issues:  (1) Will the value LSC places in ToolScratch still be
valid by the time it is retrieved?  and (2) Is LSC smashing someone
else's (e.g. the Menu Manager's) legitimate use of ToolScratch?

(1) The user is expected to retrieve the value in ToolScratch
first thing as the code resource begins to execute; there have been
no Toolbox calls since the value was set, so it must still be valid.

(2) ToolScratch is not guaranteed to be preserved across calls to
the Toolbox.  Is the Menu Manager right to assume that the
MDEF makes no Toolbox calls?  Of course not.  Therefore, the MDEF
is under no obligation not to step on ToolScratch.

Anyway, this is all moot with 3.0, but I though I'd try to set the
record straight.

-- Michael Kahl, Symantec