[comp.sys.mac.programmer] Simulating a Menu Bar with Pop-Up Menus

tim@hoptoad.uucp (Tim Maroney) (11/28/89)

Some applications, like FullWrite, give windows their own menu bars.
That is, beneath the drag rectangle, there is a horizontal bar roughly
twenty pixels high, followed by a line or double-line separator,
followed by the effective window content region.  In this window menu
bar, the names of the various menus available for that window appear in
12-point system font, and clicking in this bar works much the same as
clicking in the screen's main menu bar -- the menu title clicked in
pops up and can be selected from, and going to another title pops up
another menu.

All well and good.  But there is one problem with this -- the raw OS
doesn't give you what you need to implement this behavior.  It seems as
if it would be easy to implement with PopUpMenuSelect, and it is, but I
have not been able to do it without a trap patch.  It's within the
guidelines as much as any trap patch can be, but it still worries me
and I'd like to do without it if possible.  I wrote all the code to do
this in the last 24 hours, so I may be overlooking something that will
hit me tomorrow, but I'd appreciate advice from the net.

Here's the problem.  PopUpMenuSelect keeps the menu in place until the
button comes up.  This is not how menu bar tracking works.  Instead,
when the mouse moves up into the menu bar over a different title, a new
menu is supposed to pop up (and when the mouse moves off the top edge,
right edge or left edge of the menu bar area, the menus are supposed to
go away).  But there's no way to abort PopUpMenuSelect when this
condition is sensed.  It's easy enough to sense -- you just use the
low-memory MenuHook routine which gets called periodically while
MenuSelect or PopUpMenuSelect is active.  But MenuHook takes no
parameters and returns no arguments; it does not have any built-in way
to abort the PopUpMenuSelect.  (MBarHook does, but it only gets called
once -- in fact, I'm not sure PopUpMenuSelect even calls it once,
though MenuSelect does.)

So, what I do now is patch Button.  I save the address of the real
Button before I do menu tracking.  I store a routine called MyMenuHook
in MenuHook; this is responsible for determining whether the current
mouse position would require PopUpMenuSelect to be killed.  When
MyMenuHook determines that this has happened, it sets a Boolean flag
for my tracking loop to pick up when PopUpMenuSelect returns, then
puts the address of a routine called FakeButton into the trap table
in place of the real Button (using NSetTrapAddress) and posts a fake
mouse-up event.

PopUpMenuSelect goes to call WaitMouseUp to see if the user has
released the button.  It calls Button, which is now my fake button
routine.  FakeButton first restores the trap address of the real Button
(again using NSetTrapAddress), then returns false.  So it isn't a tail
patch; it never calls the original Button.  Since Button returned
false, WaitMouseUp pulls off the mouse-up event and returns false.
This return value signals PopUpMenuSelect to quit.  The application
checks the "aborted" Boolean flag set by MyMenuHook, and if it is true,
gets the new mouse position and restarts the tracking loop on a
different menu or no menu.  Otherwise, PopUpMenuSelect returned
normally, and the application interprets its return value as a menu
command.

This all works great, and it's not as hard as it sounds.  But it really
irks me that I have to patch Button this way.  I've never had to use
that particular devious strategy, an application-specific trap patch,
before.  It's better than calling the menu definition procedures
myself, and it works, but that's about the most I can say for it.  Does
anyone have a cleaner idea for how to implement a document window menu
bar?  Also, does anyone see a way that this could fail, as long as
PopUpMenuSelect continues to call WaitMouseUp, and WaitMouseUp
continues to call Button before checking for a mouse-up event?

(I know about the possible vanishing of low-memory globals already,
thanks.) (Also, I don't think it's safe to do a longjmp out of a trap
into an application, though if you have a different opinion, feel free
to defend it.)
-- 
Tim Maroney, Mac Software Consultant, sun!hoptoad!tim, tim@toad.com

FROM THE FOOL FILE:
"Women's wages are 56% of men's -- but that's not necessarily evidence
 of discrimination in employment."
  -- Clayton Cramer in news.groups and soc.women

lippin@spam.berkeley.edu (The Apathist) (11/28/89)

I don't think there's a good way to do this; Apple either doesn't want
it to be easy or is just being lazy.  But I've also hacked a menu bar
with PopupMenuSelect; I don't know that my way is any better than
Tim's, but here are the differences anyway:

First, I don't think that calling the MDEFs directly should be
overlooked as a possibility.  It has the advantage of being less of a
hack.  On the other hand, it is difficult and it raises some
compatibility questions (future MDEFs; drawing into the desktop).

But I didn't use this approach; instead, I put a hack into my MDEF.  I
pass it an address using a high-numbered message, up where they've
been reserved for applications.  The address lets me communicate back
and forth.  Before going into PopupMenuSelect, I tell it where the
menu bar is, and where the menu's title is.  If the MDEF finds the
mouse in the menu bar, but not in the title area and the mouse is
StillDown(), it posts a fake mouse-up-mouse-down, and sets a flag
indicating the fakeness.  PopupMenuSelect senses this, and returns to
the caller, which then grabs the mouse-down with GetNextEvent if it
was faking, and goes back to tracking the menu bar.  (The mouse down
may not be necessary, but I prefer to post mouse events in pairs, so
that the event queue will have less chance of getting out of sync with
the mouse button.)

Now, using a hacked MDEF instead of a MenuHook is a dubious advantage;
it won't suffer when low memory globals go away, but if MDEF 0 changes
in the future, my menus won't.  This is fine for me -- these menus are
nonstandard anyway, but it's a point that should be considered.

I like posting events better than patching Button, but it's a close
call.  BTW, the patch Tim describes may not be a tail patch by a
strict definition, but it suffers from the tail patch problem -- it
blocks come-from patches.  On the other hand, it blocks them only
during PopupMenuSelect, and only there on rare occasions.

Finally, I agree that longjumping out of a trap is a very bad idea
(although my code does do it in some emergencies).  It'll get you back
to your application ok, but it may leave some global resources (the
screen in this example) in an inconsistent state.

					--Tom Lippincott
					  lippin@math.berkeley.edu

 "Ask a fish head anything you want to.  It won't answer you; they can't talk."
					--"Fish Heads"

zben@umd5.umd.edu (Ben Cranston) (11/29/89)

Just thought I would mention that the November 1988 issue of MacTutor had an
article on implementing menus in windows.  Their code does muck about with
"TopMenuItem" (lowmem global?) and knows entirely too much about the insides
of Menu records. V4#11 P94 and also in Best Of IV on page 388.

-- 
Sig     DS.L    ('ZBen')       ; Ben Cranston <zben@Trantor.UMD.EDU>
* Network Infrastructures Group, Computer Science Center
* University of Maryland at College Park
* of Ulm

svc@well.UUCP (Leonard Rosenthol) (11/29/89)

In article <9095@hoptoad.uucp> tim@hoptoad.uucp (Tim Maroney) writes:
>[Menu Bars in Windows discussion]
>All well and good.  But there is one problem with this -- the raw OS
>doesn't give you what you need to implement this behavior.  It seems as
>if it would be easy to implement with PopUpMenuSelect, and it is, but I
>have not been able to do it without a trap patch.  It's within the
>guidelines as much as any trap patch can be, but it still worries me
>and I'd like to do without it if possible.  I wrote all the code to do
>this in the last 24 hours, so I may be overlooking something that will
>hit me tomorrow, but I'd appreciate advice from the net.
>
>[All about Tim's Patch]
>
>This all works great, and it's not as hard as it sounds.  But it really
>irks me that I have to patch Button this way.  I've never had to use
>that particular devious strategy, an application-specific trap patch,
>before.  It's better than calling the menu definition procedures
>myself, and it works, but that's about the most I can say for it.  Does
>anyone have a cleaner idea for how to implement a document window menu
>bar?  Also, does anyone see a way that this could fail, as long as
>PopUpMenuSelect continues to call WaitMouseUp, and WaitMouseUp
>continues to call Button before checking for a mouse-up event?
>
	I've done menuBars in windows before, and although it does pose problems
of it's own, I choose to use the 'Calling the MDEF' method.  My reasoning behind
this was that I was basically rewriting MenuSelect and that since MenuSelect
was doing it, so could I!  I've had no problem with it so far, and I doubt that
it will die anytime soon (as changing the calling conventions for MDEFs would
be a BAD MOVE on Apple's part as the world would break..).
	There was an article in MacTutor about doing this a while back, where
the author actually implemeted a complete set of 'wMenuBar' tools for doing this
and that could be easily moved from application to application. (ie nice and
generic.)  I have not look at it since it came out, but he also used the 'call
the MDEF' method (though also did some interesting things as well ;-)

	There is one other way to do window menuBars that I have seen done,
though not tried it myself, and that is to write your own MDEFs!!  Just like
a TearOff Menu may post an event when it gets torn off, your MDEF can post
events to tell you that t he mouse has moved out of the title area and that
you now need to select the next one over (and you can either send separate msgs
for right vs. left or calc it yourself).  Although it requires writing your own
MDEF, this may be the most compatible way to go since you simply call
PopUpMenuSelect as you would anyway, and there is no traps patchs and no direct
calls to defProcs.


-- 
+--------------------------------------------------+
Leonard Rosenthol        |  GEnie : MACgician
Lazerware, inc.          |  MacNet: MACgician
UUCP: svc@well.UUCP      |  ALink : D0025

jwright@atanasoff.cs.iastate.edu (Jim Wright) (11/29/89)

There is one complication that I've not seen mentioned yet.  With the
regular menu bar, you can be relatively confident it is a constant
width.  Windows can be resized.  The McSink DA is a good example of
one way to handle windows narrower than the menus.

I have no idea how it is done.

-- 
Jim Wright
jwright@atanasoff.cs.iastate.edu

tim@hoptoad.uucp (Tim Maroney) (11/30/89)

I got an answer via e-mail very quickly to my posting.  Unfortunately,
I didn't save the writer's name or address, but it was one of the
authors of a program called Nisus.  In case anyone else wants to do
this, the answer is:  MenuSelect or PopUpMenuSelect can be aborted
simply by posting a mouseUp event from MenuHook.  It will terminate as
soon as it sees a mouseUp in the queue; it will not wait for the Button
routine to return false.  Of course, this makes perfect sense, since
the mouse could have been clicked again already.  So there is no need
for a trap patch on Button.

Thanks to my correspondent for providing this information.
-- 
Tim Maroney, Mac Software Consultant, sun!hoptoad!tim, tim@toad.com

FROM THE FOOL FILE:
"As the expert on stupid, I'll take your word for it."
  -- Richard Sexton, richard@gryphon.COM

rob@cs.mu.oz.au (Robert Wallen) (12/01/89)

In article <1989Nov28.053644.28100@agate.berkeley.edu> lippin@math.berkeley.edu writes:
>I don't think there's a good way to do this; Apple either doesn't want
>it to be easy or is just being lazy.

On a related issue.  Has anyone worked out how to hack the PopUpMenuSelect so
that it doesnt present the users with that stupid blank rectangle and the
scroll arrow if it is too close to the edge of the screen.

The best I have been able to do is ask the MDEF what the size of the 
rectangle is going to be and adjust the 'initial item' that PopUpMenuSelect
will be over but have found weird behaviour with this as well.  Some pixel
positions seem to generate different rectangles to others.  Anyone out there
have any clues?

I did see a message once from someone at Symantec claiming that they had
solved the problem by some sort of stub MDEF that called the standard. However,
the last time I checked Think C still had the problem (move a window close to
the bottom of the screen and hold option down while clicking the title bar.)

oster@dewey.soe.berkeley.edu (David Phillip Oster) (12/02/89)

In article <2880@murtoa.cs.mu.oz.au> rob@murtoa.UUCP (Robert Wallen) writes:
>I did see a message once from someone at Symantec claiming that they had
>solved the problem by some sort of stub MDEF that called the standard. However,
>the last time I checked Think C still had the problem (move a window close to
>the bottom of the screen and hold option down while clicking the title bar.)

I just checked THINK C V.4 for this. It crashed with a bus error.  
It is undoubtedly a bug in Apple's code.
On the other hand, my custom MDEFs have never had an problem with this.
I've even written pop-up grid menus that can scroll left and right as well
as up and down.


> The mac is a detour in the inevitable march of mediocre computers.
> drs@bnlux0.bnl.gov (David R. Stampf)
--- David Phillip Oster          -master of the ad hoc odd hack. 
Arpa: oster@dewey.soe.berkeley.edu 
Uucp: {uwvax,decvax}!ucbvax!oster%dewey.soe.berkeley.edu 

rob@cs.mu.oz.au (Robert Wallen) (12/03/89)

>I just checked THINK C V.4 for this. It crashed with a bus error.  

Hmmm.  Its been a while, but the last time I checked (may have been on a PLUS)
it just created a real small window which had a scroll up AND a scroll down
arrow and no entries ...

>It is undoubtedly a bug in Apple's code.

Yeh?  Why?  I mean sure the standard MDEF doesnt work quite the way I want it
to but I have never had it crash with bus errors or address errors...
Id blame Symantec before Apple on this one.

>On the other hand, my custom MDEFs have never had an problem with this.

My biggest object to writing my own MDEF (which is basically a copy of MDEF-0)
is that I cant track Apples changes to the system.  I dont have a color Mac
(just an SE/30 :-) and so its 'real hard' to make sure that my custom MDEF,
CDEF, etc handle all the appropriate 'mctb' thingies.  I would prefer to be
able to call the 'standard' code because I know (hope) that Apple will keep it
up to date for me.

oster@dewey.soe.berkeley.edu (David Phillip Oster) (12/03/89)

In article <2884@murtoa.cs.mu.oz.au> rob@murtoa.UUCP (Robert Wallen) writes:
_>>I just checked THINK C V.4 for this. It crashed with a bus error.  
_>>It is undoubtedly a bug in Apple's code.

_>Yeh?  Why?  I mean sure the standard MDEF doesnt work quite the way I want it
_>to but I have never had it crash with bus errors or address errors...
_>Id blame Symantec before Apple on this one.
I owe Apple an apology. I just double checked using a pop-up menu tester
program I wrote.  Apple's MDEF doesn't crash on its own, this is a bug in
THINK C Version 4.

_>My biggest object to writing my own MDEF (which is basically a copy of MDEF-0)
_>is that I cant track Apples changes to the system.  I dont have a color Mac
_>(just an SE/30 :-) and so its 'real hard' to make sure that my custom MDEF,
_>CDEF, etc handle all the appropriate 'mctb' thingies.  I would prefer to be
_>able to call the 'standard' code because I know (hope) that Apple will keep it
_>up to date for me.

Apple's source code for the current MDEF 0 is available on some for-pay
services such as MacNet.  You can call the standard code, just call it
like you would any code handle:
	mdef0 = GetResource('MDEF', 0);
	HLock(mdef0);
	CallPascal(mesg, men, mRect, where, which, *mdef0);
	HUnlock(mdef0);
The standard warnings about tail patches apply here, and you can't just
jump to it, since you need to unlock it after, but I think the risk is
acceptable, since the menu manager has to be able to call user-written
MDEFs, so it unreasonable to expect MDEF0 to patch bugs in the system.


> The mac is a detour in the inevitable march of mediocre computers.
> drs@bnlux0.bnl.gov (David R. Stampf)
--- David Phillip Oster          -master of the ad hoc odd hack. 
Arpa: oster@dewey.soe.berkeley.edu 
Uucp: {uwvax,decvax}!ucbvax!oster%dewey.soe.berkeley.edu 

lippin@twinkies.berkeley.edu (The Apathist) (12/04/89)

Recently rob@murtoa.UUCP (Robert Wallen) wrote:
>
>On a related issue.  Has anyone worked out how to hack the PopUpMenuSelect so
>that it doesnt present the users with that stupid blank rectangle and the
>scroll arrow if it is too close to the edge of the screen.

PopupMenuSelect, when used as directed, produces the correct result --
it goes to some trouble to put the item you choose under the mouse,
and this can produce some funny looking scrolling menus if you're near
the edge of the screen.

But when you're perverting PopupMenuSelect to serve other ends, the
way menus are positioned may not be appropriate.  The best way I've
found around this is to have the MDEF misinterpret the hPopUpMsg.  For
my menu bar in a window, for example, I use this to bounce the menu
off the edge of the screen, so that it never has to scroll (givben
sufficiently short menus, of course.  LSC 4.0, on the other hand,
seems to use it to get the menu positioned properly underneath the
title bar of the window -- in extreme cases, making the menu too small
to use; Apple didn't (and had no reason to) provide for the
possibility of scrolling menus less than 3 items long.  (BTW, I
haven't been able to reproduce David Phillip Oster's bus error when
trying this on my mac II, but perhaps I haven't tried hard enough.)

					--Tom Lippincott
					  lippin@math.berkeley.edu

     "Truth is the most valuable thing we have.  Let us economize it."
			   --Mark Twain, "Pudd'nhead Wilson's New Calendar"

lim@iris.ucdavis.edu (Lloyd Lim) (12/05/89)

In article <2884@murtoa.cs.mu.oz.au> rob@murtoa.UUCP (Robert Wallen) writes:
>>I just checked THINK C V.4 for this. It crashed with a bus error.  
>
>Hmmm.  Its been a while, but the last time I checked (may have been on a PLUS)
>it just created a real small window which had a scroll up AND a scroll down
>arrow and no entries ...

I just checked this on my IIcx with 6.0.3 and it doesn't crash either.  An
INIT might have caused this crash?

>>It is undoubtedly a bug in Apple's code.
>
>Yeh?  Why?  I mean sure the standard MDEF doesnt work quite the way I want it
>to but I have never had it crash with bus errors or address errors...
>Id blame Symantec before Apple on this one.

I agree since Symantec's MDEF is doing things to Apple's MDEF that wouldn't
ever happen normally.

>>On the other hand, my custom MDEFs have never had an problem with this.
>
>My biggest object to writing my own MDEF (which is basically a copy of MDEF-0)
>is that I cant track Apples changes to the system.  I dont have a color Mac
>(just an SE/30 :-) and so its 'real hard' to make sure that my custom MDEF,
>CDEF, etc handle all the appropriate 'mctb' thingies.  I would prefer to be
>able to call the 'standard' code because I know (hope) that Apple will keep it
>up to date for me.

If the thing you are trying to achieve is not that much different from the
standard Apple supplied *DEF, total compatibility can be achieved by calling
the Apple version to handle the normal stuff.  I've had experience doing this
with my Default CDEF and it works fine.  I also get bugged by those big white
empty spaces in pop-up menus so I'm writing an MDEF to fix this.  It will
scroll better than Symantec's and I'll release it just like the Default CDEF.

+++
Lloyd Lim     Internet: lim@iris.ucdavis.edu (128.120.57.20)
              Compuserve: 72647,660
              US Mail: 146 Lysle Leach Hall, U.C. Davis, Davis, CA 95616

rob@cs.mu.oz.au (Robert Wallen) (12/05/89)

Recently lippin@math.berkeley.edu writes:
>Recently rob@murtoa.UUCP (Robert Wallen) wrote:
>>
>>On a related issue.  Has anyone worked out how to hack the PopUpMenuSelect so
>>that it doesnt present the users with that stupid blank rectangle and the
>>scroll arrow if it is too close to the edge of the screen.
>
>PopupMenuSelect, when used as directed, produces the correct result --
>it goes to some trouble to put the item you choose under the mouse,
>and this can produce some funny looking scrolling menus if you're near
>the edge of the screen.

The correct result?  So many times, I find that it scrolls the lot straight
away because I was positioned on the arrow...

>But when you're perverting PopupMenuSelect to serve other ends, the
>way menus are positioned may not be appropriate.  The best way I've

I think I dislike the term 'perverting'.  What is PopUpMenuSelect supposed to
do if its not pop-up menus where the user clicks?  Personally, I think this is
one of the arguments that Apple used to convince the world that it didnt steal
too liberally from Smalltalk -- Xerox seemed to handle popups properly 
regardless of where the mouse was clicked 8-)

>found around this is to have the MDEF misinterpret the hPopUpMsg.  For

Once again, we come down to a custom MDEF.  Apple DTS, are you reading this
string?  If I need a custom MDEF, seems to me that MDEF-0 must be broken...

>my menu bar in a window, for example, I use this to bounce the menu
>off the edge of the screen, so that it never has to scroll (given

How do you mean?  Do you miscalculate the rectangle requested?

>sufficiently short menus, of course.  LSC 4.0, on the other hand,
>seems to use it to get the menu positioned properly underneath the
>title bar of the window -- in extreme cases, making the menu too small
>to use; Apple didn't (and had no reason to) provide for the
>possibility of scrolling menus less than 3 items long.  (BTW, I

^^^^^ Apple DTS -- please take note

>haven't been able to reproduce David Phillip Oster's bus error when
>trying this on my mac II, but perhaps I haven't tried hard enough.)

Nor I on the SE/30...

lippin@ronzoni.berkeley.edu (The Apathist) (12/06/89)

Recently rob@murtoa.UUCP (Robert Wallen) wrote:
>Recently lippin@math.berkeley.edu writes:

>>PopupMenuSelect, when used as directed, produces the correct result --
>>it goes to some trouble to put the item you choose under the mouse,
>>and this can produce some funny looking scrolling menus if you're near
>>the edge of the screen.

>The correct result?  So many times, I find that it scrolls the lot straight
>away because I was positioned on the arrow...

If you found yourself positioned on the arrow, then PopupMenuSelect
wasn't used as directed.  As described in IM and the Human Interface
Guidelines, a popup menu looks like "Title: Menu Item" with a
menu-shaped box around the menu item.  When pressed, the title is
highlighted, and a menu springs up around the item.  The menu item
doesn't budge, so if you let the mouse up without moving, the same
item is selected.  This is generally only used in dialog boxes.  I
can't think of a common example of this use, because few programs use
it this way, but the folder ancestors menu in standard file has
everything but the title.

>>But when you're perverting PopupMenuSelect to serve other ends, the
>>way menus are positioned may not be appropriate.

>I think I dislike the term 'perverting'.  What is PopUpMenuSelect supposed to
>do if its not pop-up menus where the user clicks?  Personally, I think this is
>one of the arguments that Apple used to convince the world that it didnt steal
>too liberally from Smalltalk -- Xerox seemed to handle popups properly 
>regardless of where the mouse was clicked 8-)

I chose the word with care.  PopupMenuSelect is not designed for menu
bars in windows, windows that pull down from title bars of windows,
nor for Smalltalk-style menus anywhere you click.  And, as it happens,
it doesn't work perfectly for these tasks, although hacks exist to
make it work adequately.

It might be better to have a new trap which was designed for these
purposes, but I imagine that Apple is reluctant to create one when
there is no HIG-blessed use for such a thing.  Adding to the HIG is a
slow and thoughtful process, and I don't doubt that this has been, and
probably still is being considered.  In the meantime, for those of us
that can't resist, there's this hack.

>>The best way I've found around this is to have the MDEF misinterpret
>>the hPopUpMsg.

>Once again, we come down to a custom MDEF.  Apple DTS, are you reading this
>string?  If I need a custom MDEF, seems to me that MDEF-0 must be broken...

It's not broken -- it's doing what it was designed for: trying to put
the selected item in the place you chose for it.  But for this hack,
that's not what we want.

>>For my menu bar in a window, for example, I use this to bounce the
>>menu off the edge of the screen, so that it never has to scroll (given
>>sufficiently short menus, of course.

>How do you mean?  Do you miscalculate the rectangle requested?

Yes.  I return a rectangle that's entirely on the screen, and which is
large enough to hold the entire menu without scrolling.

>>LSC 4.0, on the other hand, seems to use it to get the menu positioned
>>properly underneath the title bar of the window -- in extreme cases,
>>making the menu too small to use; Apple didn't (and had no reason to)
>>provide for the possibility of scrolling menus less than 3 items long.

>^^^^^ Apple DTS -- please take note

If used in the blessed manner, the only way this should happen is if
the monitor the menu is on is less than about 80 pixels high.  Hardly
a situation Apple should lose corporate sleep over.

					--Tom Lippincott
					  lippin@math.berkeley.edu

	"Please do not defeat this important saftey feature."