instone@bgsuvax.UUCP (Keith Instone) (05/16/89)
I know it has nothing to do with poor writing (except an example),
but here goes ...
I have been thinking about all the different ways people
are _studying_ user interfaces now-a-days.
1.Lots of people are building user interface management systems (UIMSs) to
make building interfaces easier.
2.Several companies are making prototype interfaces and testing them
to see which specific style/lookNfeel is best for them.
3.Some people are doing laboratory experiments on both parts of the
interface (menus vs icons, choice of color, ...) and on complete
systems (direct manipulation vs commands vs menus, etc).
My question deals with the third. In all those papers, they never
talk about what _tools_ they used to test these things. Is that
because there are no tools supporting this? At CHI 89, 3 papers were
presented under the heading "Tools and environments for interface
DESIGN". Why not the same for "interface RESEARCH"?
For example, at CHI 88, the (in)famous Linear vs Pie menu research
was presented. It was implemented on a Sun under MIT's X window
system. That was all I could find out about HOW the test system was
implemented. So, if anyone else wants to test this, do they have to
do it from scratch? There isn't a run-on-any-machine program which
lets you control menus enough for empirical testing?
So, I guess my question for all of you is:
Has anyone made a generic tool for user interface research?
(defn. generic - runs on any system, supports any
interaction style, easily configurable)
[note: 'generic' is approximately like 'standard', but I don't
want to scare anyone away by using the s-word. (^; ]
If yes, what?
If no, why not? Too hard? Nobody wants one? Waste of time?
Inquiring minds want to know!
Keith Instone, Research Associate
Department of Computer Science, Bowling Green State University
CSNET : instone@andy.bgsu.edu
UUCP : ...!osu-cis!bgsuvax!instone
BITNET : KINSTON@TRAPPER, INSTONE@BGSUOPIErv01@GTE.COM (Robert Virzi) (05/16/89)
In article <4128@bgsuvax.UUCP>, instone@bgsuvax.UUCP (Keith Instone) writes: > I have been thinking about all the different ways people > are _studying_ user interfaces now-a-days. > > 1.Lots of people are building user interface management systems (UIMSs) to > make building interfaces easier. > > 2.Several companies are making prototype interfaces and testing them > to see which specific style/lookNfeel is best for them. > > 3.Some people are doing laboratory experiments on both parts of the > interface (menus vs icons, choice of color, ...) and on complete > systems (direct manipulation vs commands vs menus, etc). > ... stuff deleted ... > > So, I guess my question for all of you is: > > Has anyone made a generic tool for user interface research? > > (defn. generic - runs on any system, supports any > interaction style, easily configurable) > > [note: 'generic' is approximately like 'standard', but I don't > want to scare anyone away by using the s-word. (^; ] > > If yes, what? > If no, why not? Too hard? Nobody wants one? Waste of time? > > Inquiring minds want to know! > I don't know if this is what Keith had in mind, but I would like to bring up a 'new' methodology that seems to be a topic of invest- igation now. Typical UI studies in a product development environment have tended to focus on building accurate prototypes of the intended user interface that capture as much of the look and feel of the user interface as possible. Successive iterations are performed on the prototype to move it closer and closer to the 'final' product. The UI is treated as a whole, and high fidelity is sought (fidelity is measured by how close the prototype seems to be to the end result). An alternative approach is to actively avoid high fidelity to the entire interface, while testing small pieces of it. Thus the entire user interface problem is subdivided into a number of critical parts, each of which is attacked using smaller, cheaper, special purpose prototypes. The data accumulated from these smaller studies is then combined and formalized in a description of the interface as it is to be built (a requirements or specification document). Are these appraoches really different? I think that they are. Perhaps the main difference is that the latter approach, let's call it low fidelity prototyping, can occur earlier in the design process. The experimenter does not need to know what the delivery platform will be, and can use results from the early prototypes to drive the selection. (I believe this has been called requirements prototyping in the literature.) By subdividing the problem, the researcher can build cheaper prototypes that may allow him/her to explore a broader range of alternatives. Let me give a brief example. Let's say a university is going to build a system that allows students to register for courses. Under scenario 1, a group of designers chooses a delivery platform and sets about building a prototype. Let's assume they choose to use kiosks that will be placed strategically around campus. They would then set about optimizing the interface for this delivery vehicle. Under scenario 2, the designers ask the question, What's the best delivery vehicle? They might build two small, only partially functioning prototypes, that capture the essense of the two approaches. Let's say the kiosk approach is one and an audiotext system is the other. The prototypes would be, by necessity, small and only capture a piece of the total user interface. They might be mocked up using crude techniques and fidelity would be sacrificed to gain breadth of coverage. The benefit is that one avoids the trap of local optima while seeking a global one. I would draw the analogy to simulated annealing techniques applied to operations research. Which approach is better? What, if anything, can you learn using these low fidelity prototypes? If it turns out that lo-fi is a cheaper way to gain information, then it will affect the tools we need to do user interface reasearch. We would want tools that allow broad coverage, but that sacrifice depth and flexibility within a narrow domain. We need tools that will let us see the forest, not just one particular tree. So, get out your flamethrowers, I've got my asbestos suit ready. -- -Bob Virzi | Innuendo ... rv01@gte.com | ...!harvard!bunny!rv01 | ... and out the other.
instone@bgsuvax.UUCP (Keith Instone) (05/17/89)
From article <6932@bunny.GTE.COM>, by rv01@GTE.COM (Robert Virzi): > In article <4128@bgsuvax.UUCP>, instone@bgsuvax.UUCP I wrote: >> I have been thinking about all the different ways people >> are _studying_ user interfaces now-a-days. >> >> 1.Lots of people are building user interface management systems (UIMSs) to >> make building interfaces easier. >> >> 2.Several companies are making prototype interfaces and testing them >> to see which specific style/lookNfeel is best for them. >> >> 3.Some people are doing laboratory experiments on both parts of the >> interface (menus vs icons, choice of color, ...) and on complete >> systems (direct manipulation vs commands vs menus, etc). >> > ... stuff deleted ... >> > I don't know if this is what Keith had in mind, but I would like > to bring up a 'new' methodology that seems to be a topic of invest- > igation now. > - - stuff deleted about hi-fidelity and lo-fidelity prototyping - - Actually, no, this was not what I had in mind. You are dealing with studying SPECIFIC interfaces (see #2 above) where I am more interested in studying interfaces IN GENERAL (#3). But there are parralells (oops, that should be parallels). How should you study user interfaces? Look at just pieces of it (icons, commands, menus), draw conclusions and then try to incorporate your findings into a complete interface. (Is this like lo-fidelity prototyping?) The problem is trying to use the tightly-controlled results from the laboratory in the (uncontrollable) real world. Sometimes the lab stuff is far removed from the actual computer. For example, some doctoral research studying icons started with paper and pencil experiments (SIGCHI Bulletin, Oct 86). This is fine, but the REAL research has to come from more real-world situations. Why did she start with paper and pencil? There aren't any tools to support this on computers. Or, build complete (but general) interfaces and test them. Afterall, you can never be sure how things fit together until you try them together. (Hi-fidelity?) Then you are studying real-world interfaces which should hopefully give usable results. The problem here is the lack of control: sometimes all you can say is "4 out of 5 subjects surveyed recommend sugarless interfaces for their patients who chew interfaces". Your conclusions are all sugar: no meat to the results. (Enough of the analogy already!) Research studying complete interfaces has thus far been confined to comparing existing packages (I think), like word processors, because there are no tools for building general interfaces. Protoyping will help here, but it different somehow. Well, in any case, I haven't been able to find any general tools to support research. I was looking for something to let me tightly control all the little pieces of the interface while maintaining the "big picture". So far, I have to assume a "NO" to my original question. If this is so, WHY? > So, get out your flamethrowers, I've got my asbestos suit ready. I am not real good at flaming, but I'll do as-best-os I can! UGH! (oooh, sounds like a new signature) Keith Instone "I am not real good at flaming, but I'll do as-best-os I can!" - me
andrew@dgbt.uucp (Andrew Patrick) (05/19/89)
In article <4128@bgsuvax.UUCP> instone@bgsuvax.UUCP (Keith Instone) writes: ... >So, I guess my question for all of you is: > >Has anyone made a generic tool for user interface research? > > (defn. generic - runs on any system, supports any > interaction style, easily configurable) > > [note: 'generic' is approximately like 'standard', but I don't > want to scare anyone away by using the s-word. (^; ] > >If yes, what? >If no, why not? Too hard? Nobody wants one? Waste of time? An interesting question. I'm not sure what you mean by a "tool for interface research". It sounds like you mean a tool for testing interfaces. That tool is the user -- the person. The ultimate test of any user interface is the opinions and behaviour of the user. Thus, our generic "tool" is to sit people down and ask them what they think, and measure what they do. This research "tool" is rather expensive, but I'm not sure that any other "tool" will do the job. -- Andrew Patrick, Ph.D. Communications Research Centre (613) 990-4675 Department of Communications, Ottawa, Canada UUCP, INTERNET: andrew@dgbt.crc.dnd.ca PATH: utzoo!bnr-vpa!bnr-rsc!dgbt!andrew BITNET: andrew@doccrc (if all else fails)
instone@bgsuvax.UUCP (Keith Instone) (05/24/89)
From article <1085@dgbt.uucp>, by andrew@dgbt.uucp (Andrew Patrick): > In article <4128@bgsuvax.UUCP> instone@bgsuvax.UUCP (Keith Instone) writes: >> >>Has anyone made a generic tool for user interface research? >> > An interesting question. I'm not sure what you mean by a "tool for > interface research". It sounds like you mean a tool for testing > interfaces. That tool is the user -- the person. The ultimate test > of any user interface is the opinions and behaviour of the user. > Thus, our generic "tool" is to sit people down and ask them what they > think, and measure what they do. > > This research "tool" is rather expensive, but I'm not sure that any > other "tool" will do the job. > Certainly you have to test interfaces with people. The problem I am addressing (I think!) is how you get the interface in front of the people in order to test it. Why, you build the interface, of course. And you write programs to collect the data, like response time and error rate. And you conduct experiments. THAT'S A LOT OF WORK. I am sure people have tools to make this work easier. I am not sure if people have made tools for generic (application and system independent) research. Let me try to explain it better. One tool for developing interfaces are UIMSs. As best as I can fathom, they are used to design interfaces and to help in implementing specific interfaces. Are they used for testing them? Isn't the general philosophy in UIMSs that they will crank out pretty useful interfaces automatically? I don't know. Are UIMSs used extensively for testing, or do they just use findings from other research to make good interfaces? Another tool is prototyping. You prototype certain systems. Your testing will tell you what is best _for_that_situation_. The results are hard to generalize because user interfaces are so complex. Maybe prototyping is also used in more general ways, if icons are useful or not, where to add color, whatever. I don't know. Let's say I want to study menus. It has been done many times, so hopefully someone has made a tool to make this research as painless as possible. Here is a wish list of what I want in my tool: o An underlying application. It should already be programmed. Maybe a simple file manipulation system for starters. Just something for the subjects to play with. o Easy modification of the interface to control for my variables. Maybe I am comparing pull down versus pop-up menus. Or liner versus pie. Or using different phrases in the menus. All these features have been programmed before. Why do it again? o Many types of interfaces supported. So I can try direct manipulation, form fill in, sounds, whatever. We don't want to be limited to the Macintosh style, for example. o Run on lots of systems. This tool would be for generic research, so the system shouldn't matter. o Invisible data collection. Just sit the subjects in front of the interface, let them do some work. Analyze the results. o Write my paper for me. Is this too much to ask? :^) Can my wish come true? My motivation for the original question: I have read articles about experiments with interfaces. They say it was done on system X with your ordinary software development tool Y. If it took 10 graduate students 6 months to built the test system, then I am surprised anyone does research at all! Maybe it didn't. I don't know. So, to summarize, there are tools for developing interfaces. Are there software tools for testing them? Are there software tools for quickly and easily experimenting with interfaces? Keith "The good Lord must have been in a particularly good mood when He created... Missouri" - Missouri tourism film
cipher@mmm.UUCP (Andre Guirard) (05/24/89)
In article <1085@dgbt.uucp> andrew@dgbt.crc.dnd.ca (Andrew Patrick) writes: >>Has anyone made a generic tool for user interface research? >... our generic "tool" is to sit people down and ask them what they >think, and measure what they do. Perhaps what the original poster intended, was a tool to automate the measurement process. Thus if you're measuring the time it takes the user to perform some operation, and the rate of error in doing so, the "tool" would hover over your application and note when keys are pressed and whether they're the right keys. It might be difficult to make this general-purpose enough and portable enough. -- "Lalabalele talala! Callabele lalabalica falahle! | Andre Guirard Callamalala galalate! Caritalla lalabalee | cipher@3m.com kallalale poo!" | "Wake me up for | the good part."
gilbert@cs.glasgow.ac.uk (Gilbert Cockton) (05/24/89)
In article <4172@bgsuvax.UUCP> instone@bgsuvax.UUCP (Keith Instone) writes: >Why, you build the interface, of course. And you write programs >to collect the data, like response time and error rate. And >you conduct experiments. THAT'S A LOT OF WORK. Some UIMSs do have logging facilities (e.g. RAPID), as do some special purpose programming environments (e.g. NeWS). See Hartson and Dix in current ACM Computing Surveys. >One tool for developing interfaces are UIMSs. As best as I can >fathom, they are used to design interfaces and to help in implementing >specific interfaces. Are they used for testing them? Isn't the >general philosophy in UIMSs that they will crank out pretty >useful interfaces automatically? Certainly not, it is ONE philosophy, from the "programmer productivity" camp. There is no necessary commitment to high quality, user-centredness, or prototyping here (last one is impossible with a high degree of automation). There is still no consensus on what a UIMS is or should be (see Myers thesis, now published). Most definitions exclude toolkits as: a) they are not tools, they are code fragments; b) they are not complete, or even near complete, as a kit must be. on English semantics, toolkits score 0/2. On UIMS criteria: c) they do not manage, the underlying application does; d) they do not support iterative, participative design; e) they are a pig to use, badly designed (if at all designed), and require a (very) experienced programmer to program for too long. Frameworks (e.g. MacApp) are an interesting third option, which do less badly on (b), (c) and perhaps (e). They generally hide what should have been hidden in a toolkit anyway :-) >for testing, or do they just use findings from other research >to make good interfaces? I wish they were. Many toolkit and UIMS implementors do not waste time reading anything as irrelevant as human factors guidelines (e.g. SunView scrollbars). > o An underlying application. It should > already be programmed. Maybe a simple file manipulation system > for starters. Just something for the subjects to play with. If you're doing serious research you need more than one of these. > o Easy modification of the interface to control for my variables. > Maybe I am comparing pull down versus pop-up menus. Or > liner versus pie. Or using different phrases in the menus. All > these features have been programmed before. Why do it again? There is no such thing as THE pie menu. There is considerable choice, and a UIMS pitched at this level could make the wrong choice. This level of simulated logical input device (i.e. choice) is best provided as a standard library configuration, which can be copied and changed. Object-orientedness could help, but the programmer has some difficult public/private decisions. Encapsulation can be a bad thing in user interface configurations. Often you need to be as low level as NeWS to allow the range of variation which could be required across user/task combinations. > o Many types of interfaces supported. So I can try direct > manipulation, form fill in, sounds, whatever. We don't want > to be limited to the Macintosh style, for example. Probably the recipe for an unlearnable mammoth. Better to have library configurations for these, built on truly general lower level abstractions. > o Invisible data collection. Just sit the subjects in front > of the interface, let them do some work. Analyze the > results. Logging is straightforward if the input event queue/list is properly designed and accessible. Higher level logging may be required though, and this can be automated for some control abstractions (e.g. RAPID networks) > o Write my paper for me. Is this too much to ask? :^) As long as you acknowledge me, you can poach anything worthwhile from here :-) >Can my wish come true? It's close to true in some places. Unfortunately, much U.S. research is concentrating on automatic generation from high level semantic models. The desire is to get away from the 'what do we do with the next mouse click' level of UIMSs. At the moment, research is very ambitious, but some groups are still interested in lower level approaches (e.g. Myers and Szekely at CMU). My money's on the bottom up camp. No one has enough experience of user interface design to develop acceptable automatic generation of user interfaces. >So, to summarize, there are tools for developing interfaces. Are >there software tools for testing them? Are there software tools >for quickly and easily experimenting with interfaces? Use prototyping tools, including Hypermedia systems. These will constrain you, but no one's short of hypotheses in HCI at the moment. Good experiments could be done with HyperCard, and you should be able to do some logging via scripts (XCMDs anyone?). Hardware logging is also a possibility, and is used by some UK groups. Here you slip something between the keyboard and the mouse, and the computer. Very low level, but you can do some things with the byte streams. -- Gilbert Cockton, Department of Computing Science, The University, Glasgow gilbert@uk.ac.glasgow.cs <europe>!ukc!glasgow!gilbert
bds@mbunix.mitre.org (Barry D Smith) (05/24/89)
Jeffrey Fox at MITRE (jef@mitre-bedford.arpa) has developed a hyper-text program called DRUID which fully implements the Mosier/Smith user interface guidelines using Hyper Card on a Mac II. They are currently starting to port it to other platforms. It's not for sale--it's for free. Send him email if you're interested.