[comp.cog-eng] Using kinesthetic memory for human interfaces

eric@snark.UUCP (Eric S. Raymond) (06/23/88)

In article <3535@pdn.uucp>, colin@pdn.UUCP (Colin Kendall) writes:
>The experience has deepened my respect for muscular memory, and increased 
>my desire to try to incorporate "pie menus" into the product we are
>developing.  (With pie menus, the item chosen depends on the direction in 
>which the cursor is moved from some central point. Studies have shown that
>after a little practice, users of these menus no longer need to
>look at the screen while making menu selections, even complicated
>cascades of selections!) 

Very interesting. Of course pure kinesthetic memory involves less handling
of abstractions (less thinking) than the normal menu paradigm (in which you
are implicitly counting down or doing language processing each time). For most
people, even heavily visual types like me, it is even the case that it's less
effort than doing a color or texture match.

This suggests a more general question. What can we do (besides pie menus, which
I agree are neat) to maximize the extent to which the user's model of routine
operations is embedded in kinesthetic memory (NLP, anyone ;-)?).

-- 
      Eric S. Raymond                     (the mad mastermind of TMN-Netnews)
      UUCP: {{uunet,rutgers,ihnp4}!cbmvax,rutgers!vu-vlsi,att}!snark!eric
      Post: 22 South Warren Avenue, Malvern, PA 19355   Phone: (215)-296-5718

drforsey@watcgl.waterloo.edu (Dave Forsey) (06/25/88)

In article <dSUgV#2ySEgB=eric@snark.UUCP> eric@snark.UUCP (Eric S. Raymond) writes:
>In article <3535@pdn.uucp>, colin@pdn.UUCP (Colin Kendall) writes:
>>The experience has deepened my respect for muscular memory,
>>Studies have shown that
>>after a little practice, users of these menus no longer need to
>>look at the screen while making menu selections, even complicated
>>cascades of selections!) 
>
>This suggests a more general question. What can we do (besides pie menus, which
>I agree are neat) to maximize the extent to which the user's model of routine
>operations is embedded in kinesthetic memory (NLP, anyone ;-)?).
>

We've been using some sort of radial selection menus since '84, and my
personal experience indicates that even 4 or 5 ( 7 +- 2? ) levels of
hierarchy is still quite managable without much cognitive burden.
I don't know if this (subjective) feature would offset the limits that
pie menus have (i.e. it is hard to put 15 items around in a circle
and have people select easily). They seem (to me) to be an order of
magnitude easier to use than these menus in which you slide the cursor
off the item to one side to pop-up the next level of the hierarchy.

A significant speed-up occurs after using a set of menus for a period
of time.  Careful arrangement of the menu items can aid in this.
(as Callahan, Hopkins, Weiser and Shniederman (CHI '88), have noted).

I'm not sure that the raw speed increase is as important as the ease in
which they can be used. I hardly even look at the items on the menu anymore,
just the overal appearance (which differ in size according to the number
of items and the length of the words contained therein) seems to be
enough to guide my choices.  

I've had no experience with the Callahan et al style, but in our version
the sub-menu pops up when the cursor enters any part of the pie sector 
(no additional button action is required).
As an aside with regards to the automatic cursor motion question,
when the sub-menu pops up the cursor moves to accommodate the new menu
position, but since i'm operating mainly through a gesture this does
not seem to bother me. (How one might quantify this effect and test it
is not clear).

I tend to think of radial selection menus as gesture recognition
with visual feedback.  Many will remember the character recognizers that
researchers were trying a couple years back. Buxton out of the University
of Toronto had a pilot system in which the menu selections were made by
the system recognizing gestures made by the user on the screen.
Hierarchical radial selection menus have the potential to provide a
very similar function since equivalent "gestures" will take you to the
same menu item.

This ability will of course vary with the target size etc,
and some gestures will be easier to do than others (noted again by
Callahan et al.) and all these extra variables will mean that the efficacy of
the pie menus will be highly dependent on the skill of the interface
designer (this is not news).

Terry Higgins, formerly of this lab, demonstrated a paint system with
radial selection menus (that also featured mouse-ahead and couple of other
interesting ideas) at SIGGRAPH '87 last year. This year i'll probably
have a couple systems at the iris users forum that use a type of pie
menu. Not as nice as Terry's, but functional for the purposes of the
programs.

Dave Forsey
Computer Graphics Laboratory
University of Waterloo
Waterloo Ont. CANADA

lrbartram@watcgl.waterloo.edu (lyn bartram) (06/25/88)

There have been several discussions about the proposed advantages of "pie", 
or radial selection, menu systems lately in various newsgroups ( comp.cog-eng
and comp.windows.misc are two that come to mind ).  A lot of attention has
been paid to the fact (? as yet not *completely* proven) that kinesthetic
memory is a powerful aid to menu selection, often more effective than simple
conscious recognition of that which is to be selected.

I'm not debating this: we have been using some form of radial menus in various
applications in our lab for several years, and as a user myself, i find them
great.  However, the use of these tools bring a whole lot of interesting
questions to the fore : what are the most effective selection mechanisms, for
example? Slide only? Slide and click? Depress, slide and release? Double
click? At a single level, this might seem trivial, but as soon
as one wishes to take advantage of cascaded menus, it is no longer so 
immaterial.  I have noticed a claim that using radial selection in cascaded
form is far preferable to using regular popup/cascaded menus, but i am a
bit dubious about this, and would like to see some solid empirical evidence....
after all, the big bonus about radial selection, in layman's terms, is that
one uses direction rather than distance or position  to get to the selection 
point.  It seems to me that cascading imposes some of the distance constraint.

Another point is  that of form: as i mentioned a few months ago, is a circle
preferable to a {hex/oct/non}agon ?  How big does the "dead" area in the
centre have to be?  One of the applications in our lab, a paint program,
uses an octagonal( well, it may be hexagonal by now) menu, since this
means that each selectable space is big enough to serve also as a form of
*slider*:  when the selections are colours, the user can "stir" the cursor on
the selection clockwise for more saturation, and counter-clockwise for less, 
providing a very nice and intuitive interaction technique.

I know that there is some very pertinent work in ergonomics studies of
instrument positions on control panels, et al,  to do with immediate access 
to information and selection of particular tools.  Can anyone point me to
some specific references? It appears that the time has come for some
rigorous testing of radial selection techniques, and it would be nice to
know what previous studies in related fields have to tell us about the
use of kinesthetic memory in system design.

bwk@mitre-bedford.ARPA (Barry W. Kort) (06/28/88)

A variation on the pie menu would be a tic-tac-toe shaped menu,
in which the the most common menu choice would be in the middle.
(A pie menu could also include a bulls eye in the middle giving
room for one more choice.)

--Barry Kort

doug@feedme.UUCP (Doug Salot) (06/29/88)

Hey, I've got an idea: How about a paradigm in which subtle wrist and
finger movements are used to select items from a menu, and the movements
themselves map in some natural way to the meaning of item being selected.
Once the associations are learned, users will be able to select items
with their eyes closed!  But of course the hard part is trying to come
up with a universal movement->meaning map.  If only there were a way
to associate....  Hey!  What if you used some sort of mnemonic, say
a letter or group of letters that could easily map to verbs and nouns,
and the subtle movements could map directly to letters!

We could call this device the keyboard!  It could obsolete these
clumsy mice we're all trying to use for tasks they're clearly not
suited for.
-- 
Doug Salot || doug@feedme.UUCP || ...{trwrb,hplabs}!felix!dhw68k!feedme!doug
                    "Thinking: The Thinking Man's Sport"

joel@pyr.gatech.EDU (Joel Rives) (06/29/88)

>bwk@mbunix (Kort) (bwk@mbunix, <35362@linus.UUCP>):
>(A pie menu could also include a bulls eye in the middle giving
>room for one more choice.)

Yes! This bullseye position would also be very approriate for default
selection.

This also leads one to consider the advantages/disadvantages of using
concentric circles or partial circles (ie. arcs) with truncated pie
sections, which might allow for groupings of menu selections. My first
impression (purely imaginary -- of course) is that such a design would
be too busy. An actuall implementation might prove otherwise.

joel

-- 
		The thief
		  Left it behind--
		    The moon at the window.
						-Ryokan

cocksho@cs.glasgow.ac.uk (Tunde M. Cockshott) (06/29/88)

What is the standard method of sub-menus with pie-menus ?

The ones I have had experience of offer a child menu when
you leave the circular perimeter of the parent.
This system has the problem that the child covers a large
part of the parent.  This may be fine for experienced users
but for novices it makes it difficult for the user to reasses
his choice in the light of the contents of the child.  If the
desired item is not included in the child, then the user
would like to scan the parent for any other likely contenders.
The delay required to leave the child in order to make the parent 
visable is small but over a period of time would be frustrating.

An alternitive might be to 'fan out' a semi - circular menu from 
the edge of the pie section.  This would keep the whole menu
tree always visable.  A dissadvantage is that it would limit the 
number of entries in the sub menus.

Is this a problem ? What other systems are there ?


	Tunde Cockshott.

montnaro@sprite.steinmetz.ge.com (Skip Montanaro) (06/30/88)

    JR> == Joel Rives (joel@pyr.gatech.edu)
    BK> == Kort (bwk@mbunix)

    BK>(A pie menu could also include a bulls eye in the middle giving
    BK>room for one more choice.)

    JR>Yes! This bullseye position would also be very approriate for default
    JR>selection.

Suntools remembers the last item selected in a menu and makes it the default
the next time the menu is popped up. The bullseye could represent this
dynamic default, not just a static default. It is easier to implement in
some sense as well, since you don't have to decide ahead of time what the
default should be (and guess wrong most of the time).

--
Skip Montanaro (montanaro@sprite.steinmetz.ge.com, montanaro@ge-crd.arpa)

lrbartram@watcgl.waterloo.edu (lyn bartram) (06/30/88)

In article <MONTNARO.88Jun29154345@sprite.steinmetz.ge.com> <montanaro@sprite.steinmetz.ge.com> (Skip Montanaro) writes:
>Suntools remembers the last item selected in a menu and makes it the default
>the next time the menu is popped up. The bullseye could represent this
>dynamic default, not just a static default. It is easier to implement in
>some sense as well, since you don't have to decide ahead of time what the
>default should be (and guess wrong most of the time).
I would think that in many cases the default position should be a "no op"/
no selection one, to permit a user to invoke the menu without being committed
to a decision.  This brings us to the question of the "dead spot" in a pie
menu.  How big should it be?  Should there be one?  How does its size affect
the acuuracy measure of pie menu selection?  
    How does SunTools make the last menu item selected the default - by leaving
the positions of the items on the menu the same and placing the cursor on the
particular item, or by shuffling the items and having the default always in
the same place?  Both approaches could be difficult in a radial selection menu.
If direction is the selection mechanism, then changing the cursor position
counteracts it; and shuffling the position of menu items conflicts with the
principle of using kinesthetic memory.
	lyn bartram
	Computer Graphics Lab
	University of Waterloo
	Waterloo, Ontario, Canada

petree@gaigan.cs.utexas.edu (Mitch Petree) (07/04/88)

In article <5034@watcgl.waterloo.edu>, lrbartram@watcgl.waterloo.edu (lyn bartram) writes:
> In article <MONTNARO.88Jun29154345@sprite.steinmetz.ge.com> <montanaro@sprite.steinmetz.ge.com> (Skip Montanaro) writes:
> >Suntools remembers the last item selected in a menu and makes it the default
> >the next time the menu is popped up. The bullseye could represent this
> >dynamic default, not just a static default. It is easier to implement in
> >some sense as well, since you don't have to decide ahead of time what the
>
> >default should be (and guess wrong most of the time).
> I would think that in many cases the default position should be a "no op"/
> no selection on e, to permit a user to invoke the menu without being committed
> to a decision.   ...
> does SunTools make the last menu item selected the default - by leaving
> the positions of the items on the menu the same and placing the cursor on the
> particular item, or by shuffling the items and having the default always in
> the same place?  Both approaches could be difficult in a radial selection menu.
> If direction is the selection mechanism, then changing the cursor position
> counteracts it; and shuffling the position of menu items conflicts with the
> principle of using kinesthetic memory.
> 	lyn bartram

1) I prefer SunTools method of using the last item chosen in a menu as the
	default next time the menu is called up.  I have found that locality
	of reference applies in these situations (my experience only).
2) SunTools implements this by keeping the items in the same order and placing
	the cursor on the correct item.
3)  This may be harder to do on a pie menu.  I am not sure how pie menus
	work.  What kind of selector do you use?  Is it a knob that is turned
	or do you just have a pie shape and move the cursor into a slice?
	I can't see using a mouse to get direction except for direction of
	movement, whereas it sounds like you are talking about a static
	direction, i.e., the direction it is "looking".
    Just having the pie shape would be great because on the average you have
	to move a smaller distance to get to your selection.
4) I think the dead spot should be just that, a dead spot, like the title
	bar in a menu group.  It could identify the menu group or be empty
	but would not invoke any function.

	-mitch petree (Univ. Texas @ Austin, Elec. & Comp. Engr.)

rwl@uvacs.CS.VIRGINIA.EDU (Ray Lubinsky) (07/07/88)

In article <5034@watcgl.waterloo.edu>, lrbartram@watcgl.waterloo.edu (lyn bartram) writes:
> How does SunTools make the last menu item selected the default - by leaving
> the positions of the items on the menu the same and placing the cursor on the
> particular item, or by shuffling the items and having the default always in
> the same place?

The ordering of the menu remains constant, but when it pops up your cursor is
left over the last selection so that all you have to do is release the mouse
button to choose that selection again.

Is this a benefit?  Sometimes, but in general (when you need to select
something else) it takes a little cognitive effort to rescan the list from the
top and either move up or down relative to your current position to make the
other selection.

-- 
| Ray Lubinsky,                    UUCP:      ...!uunet!virginia!uvacs!rwl    |
| Department of                    BITNET:    rwl8y@virginia                  |
| Computer Science,                CSNET:     rwl@cs.virginia.edu  -OR-       |
| University of Virginia                      rwl%uvacs@uvaarpa.virginia.edu  |