palan-t@acsu.buffalo.edu (thiagarajan palanivel) (06/24/91)
I am currently working on my dissertation (tentatively) entitled : "Individual Differences in Cognition : Implications for Interface Design." I have followed some of the discussion in this group with regard to the big gap between research done in academia, and the design principles being practiced in industry. I am about six months away from finishing up and felt that I should take this oppurtunity to try and narrow this gap as much as possible, at least for my own work ! A summary of the main ideas follows. I would be grateful for any comments / criticism / suggestions / references. Especially with regard to the validity / need for such research, and the development of interface prototypes. To give some idea of the inter-disciplinary nature of this project and to give credit to the other people involved in this work: I have an undergraduate degree in engineering and a graduate degree in Operations Research. My co-advisors (and their backgrounds) are Valerie Shalin (Cognitive Psych.), and Martin Helander (Ergonomics). Erwin Segal (Cognitive Psych., Linguistics) is the third member of my committee. Raj Palanivel Dept. of Industrial Engg. SUNY @ Buffalo (716) 636-2357 --- While there has been considerable research and development in the areas of : 1) Designing interfaces to optimize performance on specific tasks/applications and 2) Designing interfaces to optimize the performance of Expert/Novice users, there has not been a whole lot of research looking at whether individual differences in cognitive processing (other than those based on the user's knowledge and/or experience) play a significant role in predicting performance. And if they do, how the effects of such differences interact with the effects ofinterface characteristics. Almost all of the previous work in this area has considered specific task domains (e.g. text-editing). Egan (1988, Chapter on Individual Differences in the "Handbook of Human-Computer Interaction", M. Helander (Ed.)) has done a comprehensive review of the area, and has tabulated the results from several studies. He points out the need to consider individual differences (quantitatively shows the highly significant effects of such differences on performance), andexplains some of the design/training methods followed in the past to accomodate such differences. My own work is focussed on whether there exist any pairings between users' cognitive profiles (the combimation of cognitive processing abilities and capacities of a user) and the INTERACTION STYLE/PRINCIPLE of an interface (e.g. Command Language based, or Direct Manipulation based), that optimize performanceat the "operating system level" (domain-inspecific tasks like manipulating and managing files, and navigating through file hierarchies). For example, can one predict that a user with high verbal and reasoning skills but relatively low spatial skills, will perform (the same task) better on a Command Language based interface, than on a Direct Manipulation based interface. From a different pointof view, can one say for example, that verbal ability is the single best predictor of performance on a command language interface ? I have done most of the theoretical development for my dissertation, and have finalized the scope, and defended my proposal. I have already 1) Classified existing interfaces into categories for the purpose of this study, and compiled lists of advantages and disadvantages for each category. 2) Identified the dimensions on which these categories differ, and selected those that are to be considered in this study. 3) Chosen three of categories ( Command Language, Menu, Direct Manipulation) that encompass sufficient variability on each of the dimensions. 4) Designed a suitable task scenario and tasks to be used in experimental tests. 5) Developed (not yet perfected) an "ad-hoc" (in that it is not very rigorous) method of extracting the cognitive skills required to perform a task from an analysis of the task (assuming it is optimally performed on a particular interface). I felt that none of the existing methods of cognitive analysis ... (thats a long story, will get into it some other time if anyone is interested). 6) Decided which cognitive abilities to consider(using tests from the ETS kit of factor-referenced cognitive tests). 7) Designed an experiment, and developed the experimental procedure. I am currently working on : 1) Finalizing the experimental tasks and perfrecting the method for extracting cognitive requirements, 2) Developing Command Language, Menu (walk-through), and Direct Manipulation interface prototypes for the experiment. I am using HYPERCARD on a MAC IIci. 3) Getting some kind of specialist support for the development of the interface prototypes. I feel that the closeness of the prototypes to existing systems willbe a major factor in determining the validity of the results of this study. Path: palan-t Newsgroups:comp.human-factors Distribution: world Followup-To: From: palan-t@minnie.eng.buffalo.edu (raj palanivel) Reply-To: palan-t@minnie.eng.buffalo.edu, v124r7s4@ubvmsb.cc.buffalo.edu (raj palanivel) Organization: SUNY Buffalo, Dept. of IE Subject:Individual Differences I am currently working on my dissertation (tentatively) entitled : "Individual Differences in Cognition : Implications for Interface Design." I have followed some of the discussion in this group with regard to the big gap between research done in academia, and the design principles being practiced in industry. I am about six months away from finishing up and felt that I should take this oppurtunity to try and narrow this gap as much as possible, at least for my own work ! A summary of the main ideas follows. I would be grateful for any comments / criticism / suggestions / references. Especially with regard to the validity / need for such research, and the development of interface prototypes. To give some idea of the inter-disciplinary nature of this project and to give credit to the other people involved in this work: I have an undergraduate degree in engineering and a graduate degree in Operations Research. My co-advisors (and their backgrounds) are Valerie Shalin (Cognitive Psych.), and Martin Helander (Ergonomics). Erwin Segal (Cognitive Psych., Linguistics) is the third member of my committee. Raj Palanivel Dept. of Industrial Engg. SUNY @ Buffalo (716) 636-2357 --- While there has been considerable research and development in the areas of : 1) Designing interfaces to optimize performance on specific tasks/applications and 2) Designing interfaces to optimize the performance of Expert/Novice users, there has not been a whole lot of research looking at whether individual differences in cognitive processing (other than those based on the user's knowledge and/or experience) play a significant role in predicting performance. And if they do, how the effects of such differences interact with the effects ofinterface characteristics. Almost all of the previous work in this area has considered specific task domains (e.g. text-editing). Egan (1988, Chapter on Individual Differences in the "Handbook of Human-Computer Interaction", M. Helander (Ed.)) has done a comprehensive review of the area, and has tabulated the results from several studies. He points out the need to consider individual differences (quantitatively shows the highly significant effects of such differences on performance), andexplains some of the design/training methods followed in the past to accomodate such differences. My own work is focussed on whether there exist any pairings between users' cognitive profiles (the combimation of cognitive processing abilities and capacities of a user) and the INTERACTION STYLE/PRINCIPLE of an interface (e.g. Command Language based, or Direct Manipulation based), that optimize performanceat the "operating system level" (domain-inspecific tasks like manipulating and managing files, and navigating through file hierarchies). For example, can one predict that a user with high verbal and reasoning skills but relatively low spatial skills, will perform (the same task) better on a Command Language based interface, than on a Direct Manipulation based interface. From a different pointof view, can one say for example, that verbal ability is the single best predictor of performance on a command language interface ? I have done most of the theoretical development for my dissertation, and have finalized the scope, and defended my proposal. I have already 1) Classified existing interfaces into categories for the purpose of this study, and compiled lists of advantages and disadvantages for each category. 2) Identified the dimensions on which these categories differ, and selected those that are to be considered in this study. 3) Chosen three of categories ( Command Language, Menu, Direct Manipulation) that encompass sufficient variability on each of the dimensions. 4) Designed a suitable task scenario and tasks to be used in experimental tests. 5) Developed (not yet perfected) an "ad-hoc" (in that it is not very rigorous) method of extracting the cognitive skills required to perform a task from an analysis of the task (assuming it is optimally performed on a particular interface). I felt that none of the existing methods of cognitive analysis ... (thats a long story, will get into it some other time if anyone is interested). 6) Decided which cognitive abilities to consider(using tests from the ETS kit of factor-referenced cognitive tests). 7) Designed an experiment, and developed the experimental procedure. I am currently working on : 1) Finalizing the experimental tasks and perfrecting the method for extracting cognitive requirements, 2) Developing Command Language, Menu (walk-through), and Direct Manipulation interface prototypes for the experiment. I am using HYPERCARD on a MAC IIci. 3) Getting some kind of specialist support for the development of the interface prototypes. I feel that the closeness of the prototypes to existing systems willbe a major factor in determining the validity of the results of this study. Path: palan-t Newsgroups:comp.human-factors Distribution: world Followup-To: From: palan-t@minnie.eng.buffalo.edu (raj palanivel) Reply-To: palan-t@minnie.eng.buffalo.edu, v124r7s4@ubvmsb.cc.buffalo.edu (raj palanivel) Organization: SUNY Buffalo, Dept. of IE Subject:Individual Differences I am currently working on my dissertation (tentatively) entitled : "Individual Differences in Cognition : Implications for Interface Design." I have followed some of the discussion in this group with regard to the big gap between research done in academia, and the design principles being practiced in industry. I am about six months away from finishing up and felt that I should take this oppurtunity to try and narrow this gap as much as possible, at least for my own work ! A summary of the main ideas follows. I would be grateful for any comments / criticism / suggestions / references. Especially with regard to the validity / need for such research, and the development of interface prototypes. To give some idea of the inter-disciplinary nature of this project and to give credit to the other people involved in this work: I have an undergraduate degree in engineering and a graduate degree in Operations Research. My co-advisors (and their backgrounds) are Valerie Shalin (Cognitive Psych.), and Martin Helander (Ergonomics). Erwin Segal (Cognitive Psych., Linguistics) is the third member of my committee. Raj Palanivel Dept. of Industrial Engg. SUNY @ Buffalo (716) 636-2357 --------------- palan-t@minnie.eng.buffalo.edu, v124r7s4@ubvmsb.cc.buffalo.edu (raj palanivel) --------------- While there has been considerable research and development in the areas of : 1) Designing interfaces to optimize performance on specific tasks/applications and 2) Designing interfaces to optimize the performance of Expert/Novice users, there has not been a whole lot of research looking at whether individual differences in cognitive processing (other than those based on the user's knowledge and/or experience) play a significant role in predicting performance. And if they do, how the effects of such differences interact with the effects ofinterface characteristics. Almost all of the previous work in this area has considered specific task domains (e.g. text-editing). Egan (1988, Chapter on Individual Differences in the "Handbook of Human-Computer Interaction", M. Helander (Ed.)) has done a comprehensive review of the area, and has tabulated the results from several studies. He points out the need to consider individual differences (quantitatively shows the highly significant effects of such differences on performance), andexplains some of the design/training methods followed in the past to accomodate such differences. My own work is focussed on whether there exist any pairings between users' cognitive profiles (the combimation of cognitive processing abilities and capacities of a user) and the INTERACTION STYLE/PRINCIPLE of an interface (e.g. Command Language based, or Direct Manipulation based), that optimize performanceat the "operating system level" (domain-inspecific tasks like manipulating and managing files, and navigating through file hierarchies). For example, can one predict that a user with high verbal and reasoning skills but relatively low spatial skills, will perform (the same task) better on a Command Language based interface, than on a Direct Manipulation based interface. From a different pointof view, can one say for example, that verbal ability is the single best predictor of performance on a command language interface ? I have done most of the theoretical development for my dissertation, and have finalized the scope, and defended my proposal. I have already 1) Classified existing interfaces into categories for the purpose of this study, and compiled lists of advantages and disadvantages for each category. 2) Identified the dimensions on which these categories differ, and selected those that are to be considered in this study. 3) Chosen three of categories ( Command Language, Menu, Direct Manipulation) that encompass sufficient variability on each of the dimensions. 4) Designed a suitable task scenario and tasks to be used in experimental tests. 5) Developed (not yet perfected) an "ad-hoc" (in that it is not very rigorous) method of extracting the cognitive skills required to perform a task from an analysis of the task (assuming it is optimally performed on a particular interface). I felt that none of the existing methods of cognitive analysis ... (thats a long story, will get into it some other time if anyone is interested). 6) Decided which cognitive abilities to consider(using tests from the ETS kit of factor-referenced cognitive tests). 7) Designed an experiment, and developed the experimental procedure. I am currently working on : 1) Finalizing the experimental tasks and perfrecting the method for extracting cognitive requirements, 2) Developing Command Language, Menu (walk-through), and Direct Manipulation interface prototypes for the experiment. I am using HYPERCARD on a MAC IIci. 3) Getting some kind of specialist support for the development of the interface prototypes. I feel that the closeness of the prototypes to existing systems willbe a major factor in determining the validity of the results of this study.
dent@DIALix.oz.au (Andrew Dent) (06/24/91)
In <80905@eerie.acsu.Buffalo.EDU> palan-t@acsu.buffalo.edu (thiagarajan palanivel) writes: >My own work is focussed on whether there exist any pairings between users' >cognitive profiles (the combimation of cognitive processing abilities and >capacities of a user) and the INTERACTION STYLE/PRINCIPLE of an interface (e.g. Command Language based, or Direct Manipulation based), that optimize performanceat the "operating system level" (domain-inspecific tasks like manipulating and >managing files, and navigating through file hierarchies). For example, can one >predict that a user with high verbal and reasoning skills but relatively low >spatial skills, will perform (the same task) better on a Command Language based interface, than on a Direct Manipulation based interface. From a different pointof view, can one say for example, that verbal ability is the single best >predictor of performance on a command language interface ? Speaking in a non-scientific way, I have a concern that your definitions of style/principle will not test the true power of direct manipulation as used by "power users". Most Mac applications I've used have a wealth of command-key combinations and shift-click/option-click/command-click etc. mouse actions. The skills or type of cognitive profile which makes a user extremely productive on a CLI is (IMHO) directly applicable to the use of these "power- user" facilities. These facilities often lack visual feedback that you are in a "mini-mode" (unlike selection of graphics tools from a palette, or click and drag to get some rubberbanded outline). Without testing this kind of environment you run the risk of comparing the "athletes" of the CLI against the "joggers" of Direct Manipulation. public comment invited... Andy Dent A.D. Software phone 09 249 2719 Mac & VAX programmer 94 Bermuda Dve, Ballajura dent@DIALix.oz Western Australia 6066 dent@DIALix.oz.au (international)
palan-t@acsu.buffalo.edu (thiagarajan palanivel) (06/24/91)
Andy Dent writes: >Speaking in a non-scientific way, I have a concern that your definitions of >style/principle will not test the true power of direct manipulation as used >by "power users". Most Mac applications I've used have a wealth of command-key >combinations and shift-click/option-click/command-click etc. mouse actions. I think that is a very valid point. However, theoretically, I am working at a slightly higher level of abstraction than just style/principle. With the vast and diverse types of interfaces currently available, it would be difficult for me to test all of them, or to test enough of them to arrive at results generalizable to all of them. What I have tried to do instead (to make any results I may get more generalizable, especially to interfaces not considerered in this study) is to list the dimensions on which interfaces differ. Then I havechosen some dimensions to study, and selected three types of interfaces that encompass a broad range on each of the dimensions. For example, I am considering the dimensions of Orientation (Object vs. Process), Mode of Dialog (Verbal vs. Physical), and Control of Dialog (User vs. System). I am not considering others such as Data representation (Hierarchical vs. Gen. Network (as in Hypermedia)), and Multi-tasking capacity. That is why I have chosen to study three interfaces which, while they may be well short of the state-of-the-art, mark some clear distinctions with respect to the interests of my work. Of course, there are dis-advantages to this approach, chief among them being the loss of face validity for this study (You looked at such ancient interfaces ????, we have come a long way since then, you know !!). >The skills or type of cognitive profile which makes a user extremely >productive on a CLI is (IMHO) directly applicable to the use of these "power- >user" facilities. These facilities often lack visual feedback that you are >in a "mini-mode" (unlike selection of graphics tools from a palette, or click >and drag to get some rubberbanded outline). >Without testing this kind of environment you run the risk of comparing the >"athletes" of the CLI against the "joggers" of Direct Manipulation. Again, I must agree that you have an extremely valid point. I am however going to have two groups of subjects, novice, and expert, so as to isolate the effects of experience and relevant knowledge. Since this is a University setting, I hope to get subjects who will fit the very tight constraints I propose to use while selecting subjects for either category. The effects of interface type, cognitive abilities, and the interaction of the two, within a group of subjects, will therefore be relatively free of the "athletes" to "joggers" comparison effects. Thanks to Andy Dent for the feedback. Comments ? Raj Palanivel Dept. of IE SUNY @ Buffalo
mcgregor@hemlock.Atherton.COM (Scott McGregor) (06/25/91)
thiagarajan palanivel writes: > For example, can one > predict that a user with high verbal and reasoning skills but relatively low > spatial skills, will perform (the same task) better on a Command Language > based interface, than on a Direct Manipulation based interface. From a > different pointof view, can one say for example, that verbal ability is the > single best predictor of performance on a command language interface ?... > 3) Chosen three of categories ( Command Language, Menu, Direct Manipulation) > that encompass sufficient variability on each of the dimensions. Will people be doing the same task different ways? How do you correct for the fact that the number of possible errors that a user can make are large in a command language (typographical errors, choosing missing objects, etc.) which are not possible in more constrained menu or direct manipulation systems (e.g. no typographical errors possible in menus, since there is nothing to type, no selection of missing objects possible in direct manipulation because they aren't presented)... Or is tolerance for errors part of the cognitive skills that your are trying to compare against? Scott McGregor Atherton Technology mcgregor@atherton.com
kirlik@chmsr.gatech.edu (Alex Kirlik) (06/25/91)
In <80905@eerie.acsu.Buffalo.EDU> palan-t@acsu.buffalo.edu (thiagarajan palanivel) writes: > >>My own work is focussed on whether there exist any pairings between users' >>cognitive profiles (the combimation of cognitive processing abilities and > For example, can one >>predict that a user with high verbal and reasoning skills but relatively low >>spatial skills, will perform (the same task) better on a Command Language based interface, than on a Direct Manipulation based interface. From a different pointof view, can one say for example, that verbal ability is the single best Possibly. You might want to be careful about the "task" you select, though. The relative contributions of the underlying task and the interface in making a task predominantly "verbal" or "spatial" is not really known. For example, high spatial ability people have been found to be superior to low spatial's on text editing (perhaps suprisingly). The offered explanation is that the user needs good spatial abilities to maintain an understanding of layout of the entire document, based on a limited access through a window (the current screen). That is, knowing "that's up there," "that's down there." If you're interested in the contribution of the interface, rather than the underlying task, you need to be careful about your experimental design (e.g., cross type (verbal/spatial) of task with type of interface. As I said, we don't yet know how depth (task) and surface (interface) properties contribute in determining what is a mostly spatial and what is a mostly verbal task. But that may be one of the contributions you could make. Good luck. Alex UUCP: kirlik@chmsr.UUCP {backbones}!gatech!chmsr!kirlik INTERNET: kirlik@chmsr.gatech.edu
mcgregor@hemlock.Atherton.COM (Scott McGregor) (06/25/91)
In article <1063@DIALix.oz.au>, dent@DIALix.oz.au (Andrew Dent) writes: > Without testing this kind of environment you run the risk of comparing the > "athletes" of the CLI against the "joggers" of Direct Manipulation. Excellent point. I have seen many people try to argue that a CLI is better for a particular task than a GUI. On the face, this argument is absurd, since there is no requirement that a GUI not accept typed commands as well as mouse moves, clicks, drags and drops. While many GUIs today IGNORE character input when and where they are expecting mouse interactions--they do not have to, and many interesting UIs get the benefits of both merely by raising a hidden CLI window whenever unexpected text input is typed. A pure CLI can therefore be seen as merely a degenerate case of a GUI that does nothing with its WIMP capabilities! Scott McGregor Atherton Technology mcgregor@atherton.com
palan-t@acsu.buffalo.edu (thiagarajan palanivel) (06/27/91)
Scott McGregor writes: >Will people be doing the same task different ways? How do you correct >for the fact that the number of possible errors that a user can make are >large in a command language (typographical errors, choosing missing >objects, etc.) which are not possible in more constrained menu or direct >manipulation systems (e.g. no typographical errors possible in menus, >since there is nothing to type, no selection of missing objects possible >in direct manipulation because they aren't presented)... Or is tolerance >for errors part of the cognitive skills that your are trying to compare >against? >Scott McGregor >Atherton Technology >mcgregor@atherton.com Yes, people will be doing the same task in different ways. The essential issue in this research is the commission of (and recovery from) errors. On the one hand, there is the issue (expressed in your post) of possibly trading off potential for error with "power". Yes, there is greater potential for error when using command languages (when compared to Menus and Direct Manipulation), but there is also, as you put it, less "constraint" in terms of the actions the user can perform, and the objects that these actions can be performed on. The real issue in this case is, is this trade-off the same for all users. Are all users equally error-prone in each environment ? If they are not, then the trade-off doesn't have the same relative magnitude for all users. There may be therefore, an interface that optimizes this balance for a particular user. And the optimal interface for another user may be different. If such unique optimal designs exist for each user, the cognitive profile of the user must play a significant (if not the sole) role in their determination. Incidentally, I have met several people who do not accept that such a trade-off exists. Rather, they feel that there is relatively little difference in the error potential of different interfaces. The errors (or even relative speeds of performance of actions in different modes) that are possible are just different in nature, and are directly dependent on the nature of the interface. Such an argument is based on the classical breakdown of errors into the categories of slips and mistakes (a la James Reason and Donald Norman), and the cognitive analysis of errors in each category. Personally, I feel that this method of error classification itself is not comprehensive enough, but that's another issue. On the other hand, and perhaps much more important to the validity of this work, previous research shows beyond a doubt that the greatest source of variability in performance (in terms of time to perform a task) between users is the commission and recovery from errors. Without the effects of errors (error-free task performance time), a 2:1 ratio usually covers the whole range of subjects' performance in experiments. With error times included, the ratio of best-to-worst performance is commonly in the 10:1 to 50:1 range for such tasks as text-editing, programming, and information search. (Egan, 1988 (referenced in my initial post) covers this aspect extensively). At a more fundamental level, I think that these results indicate that while perceptual limitations may be the limiting factor for users in general, cognitive limitations account for most of the variability between users. Raj Palanivel Dept. of IE SUNY@Buffalo (716)-636-2357