tzippy@dasys1.uucp (Tzipporah BenAvraham) (05/31/90)
Index Number: 8527 this is from Gregg vanderheiden on sensory impairments Sensory Impairment Focus Area Individuals with sensory impairments, either visual or auditory, can experience difficulty in using standard commercially available computers and other electronic devices. Since the types of needs and the degree of severity of current access problems are different for users with visual impairments than for users with hearing impairments, the two groups are addressed by separate research and development programs. Visual impairments For individuals with visual impairments, the primary needs deal with provision of some mechanism to perceive the contents of visual displays. Because most computer software relies almost exclusively on the visual display of information, the barriers faced by persons with visual impairments, particularly individuals who are severely visually impaired or blind, are quite large. Moreover, the recent trend toward increased detail in visual displays, both in the form of text formatting and graphic images, is greatly complicating the process of providing alternate access to this information. Over 29 different aids and techniques are available for providing access to the current generation of character-based screen displays, and there are several aids that can provide access to the newer graphic-based screens for those with low vision. However, there are currently no aids or techniques for providing access to the newer computer systems and their operating systems for severely visually impaired or blind individuals. The primary concern is the unavailability of screen information in a form that can be easily accessed and interpreted by special Braille, voice, or tactile display systems. While it previously was possible for special aids to access sections of memory and directly read the text that was printed on the screen, modern systems only maintain a bit image representation of the visual screen image. Furthermore, even if the text on the screen were provided (or reconstructed using optical character recognition), the full information content of the screen may not be discernible, since a significant portion of the information on the screen is often presented in pictorial or graphic form, or through the formatting, arrangement and location of text. This problem is compounded by the fact that some of the major new operating systems by Apple, Microsoft, and IBM use icons and overlapping images. These make interpretation by external devices extremely difficult if not impossible without some cooperation from the computer and operating system composing the image. In addition to the difficulties in getting access to the screen image, there aren't good strategies for presenting this information to a blind individual. Better techniques are needed for handling the diverse formatting of text (style, font, size, and spatial arrangement) if these new screens are to be properly interpreted. In addition, there is a need to identify and develop specific strategies for allowing blind individuals to directly explore charts, diagrams, and other graphics that are being used at a rapidly increasing rate in standard business reports and software. The primary needs in the area of visual impairment at the present time are: 1) the identification of techniques for tapping the display image in the new generation hardware and operating systems (for use by image enlargement and, someday, artificial intelligence-based screen interpretation systems); 2) the identification of strategies for securing the information displayed on the screen before it is turned into a visual image (in order to provide screen access techniques and to develop direct interpretation strategies for blind individuals); 3) research into the optimum techniques and strategies for blind individuals to work with formatted text displays; 4) research into the best formats and techniques for allowing blind individuals to perceive and manipulate graphic images (in order to allow them to operate the new generation programs and operating systems). Hearing impairments Individuals with hearing impairments do not experience much difficulty in dealing with current computer and software designs. They will face increasing barriers, however, if the use of sound (tones or voice) to transmit information is increased. The primary need in the area of hearing impairments is therefore in identifying where these barriers are likely to arise, and in identification of alternatives for presenting this auditory information. Most of these activities are associated with design consideration initiatives, described under the Cross-Impairment Focus Area. A particularly important consideration is the development of a +hearing impairment flag+ for operating systems. To help coordinate the efforts in this area, Trace Center staff recently participated in both a state-of-the-art conference on deafness and hearing impairment in El Paso and the Gallaudet state-of-the-art conference on telecommunications. Program plan The major program thrusts in the Sensory Impairment Area are: 1) development of blind user interfaces for graphic-based computer systems; 2) identification and development of system hooks to allow access by blind individuals; and 3) development of auditory access techniques for deaf and hearing impaired users. In addition, as with the Movement Impairment Area, efforts in this area are also directed toward a number of the projects described in the Cross-Impairment Focus Area. State-of-the-Art Planning Workshop on Access to Graphics- Based Computers by Blind Users Project Team: Charles C. Lee, M.S.; Gregg C. Vanderheiden, Ph.D.; Christine Thompson, B.S. Background The greatest single technical obstacle to computer access for blind individuals to arise in the past five years is the emergence of computers which use graphical displays as a standard part of their operating systems. The existing computer access systems for blind users (both braille-output and voice-output) rely upon a character-based display: one consisting primarily of alphanumeric characters which are stored in memory as characters rather than as pixel images. Current access systems rely on the ability to recognize characters and determine their position by directly reading the screen memory. This information can be sent directly to a speech synthesizer or braille display and there translated into an intelligible form. In graphical operating systems, screen memory stores pixel images, providing no source of text to send to a synthesizer or braille display. Furthermore, information about text position is not stored in the absolute form in which it is stored in character-based systems. The result of this situation is that there is currently no commercially available system for voice output access to graphic operating systems; and only one tactile system, displaying images of letters rather than true braille. There are many researchers and developers who have tackled these access problems, and many users with a wealth of experience in using current systems. However, information sharing has primarily occurred only through individual contact and publications. Approach In order to ascertain the state of the art in access to graphic operating systems for blind users, it was decided to host a planning workshop. This session would bring together those with a great deal of knowledge and experience to discuss the problem, to share information, and to determine research and development priorities. The workshop was scheduled to last three and a half days, to ensure that enough time was provided for sharing information and for discussion of issues. The four principle objectives were: 1) to acquaint all of the key individuals in the field with state-of-the-art information and ideas; 2) to identify promising access strategies; 3) to develop a recommended plan of action for addressing access problems of blind and visually impaired individuals; and 4) to identify and form collaborative links among individuals working in various areas. Progress The state-of-the-art planning workshop was held in Madison on October 4-7, 1988. Thirty-seven participants were invited, including representatives from computer companies, special access equipment manufacturers, consumers, and researchers. Eleven of the 32 participants were blind or deaf-blind. Pre- and post-workshop papers were solicited. The pre- workshop papers were collected and distributed to the participants prior to the conference. Twelve post-workshop papers were commissioned. These are currently being completed and assembled as a post-conference summary document, which should be released in early 1990. The report will be available to participants and other interested parties through the Trace Center Reprint Service. The workshop also resulted in several cooperative projects being launched. Publications Lee, C. C., & Vanderheiden, G. C. (1989, June). Access to graphical computers by blind users: Results of a planning workshop. In Proceedings of the 12th Annual Conference of the Association for the Advancement of Rehabilitation and Assistive Technology (RESNA). New Orleans, LA. Lee, C. C., & Vanderheiden, G. C. (1988, June). Accessibility of graphically based user interface computer systems for individuals with visual impairments. In Proceedings of the International Conference of the Association for the Advancement of Rehabilitation Technology (ICAART). Canada: Montreal. List of participants Adams, Frank R. (Special Needs System Development, IBM Entry Systems Division) Bach-y-Rita, Paul, MD (Dept. of Rehab. Medicine, University of Wisconsin-Madison) Barello, Larry (Microsoft Corporation) Blazie, Deane (Blazie Engineering) Boyd, Larry H. (Berkeley System Design, Inc.) Boyer, John (Computers to Help People, Inc.) Brabyn, John (Smith-Kettlewell Institute) Chong, Curtis (Minneapolis, MN) Cranmer, Tim (Frankfort, KY) de l'Aune, William (Rehabilitation R&D Center, Veterans Administration Medical Center) Durre, Karl P. (Computer Science Department, Colorado State University) Ewers, Neil (St. Paul, MN) Foulke, Emerson (Perceptual Alternatives Laboratory, University of Louisville, KY) Fowle, Tom (Smith-Kettlewell Institute) Gabias, Paul (Pueblo, CO) Goodrich, Gregory (Western Blind Rehabilitation Center, Veterans Administration Medical Center, Palo Alto, CA) Gunderson, Jon (Trace R&D Center) Holladay, David (Raised Dot Computing) Kasday, Leonard R.(AT&T Bell Laboratories) Milewski, Allen (AT&T Bell Laboratories) Lauer, Harvey (Blind Center) Lee, Charles C. (Trace R&D Center) Lewis, Paul (Telesensory Systems, Inc.) Mansoir, David (Sight Center, Cleveland Society for the Blind) McKinley, Jan (Western Blind Rehabilitation Center, Veterans Administration Medical Center, Palo Alto, CA) Melrose, Sue (New Berlin, WI) Millar, Susanna (Department of Experimental Psychology, University of Oxford, England) Navy, Caryn (Raised Dot Computing) Orman, Steve (Dept. of Rehab. Medicine, University of Wisconsin-Madison) Parreno, Antonio (Hospital Ramon y Cajal, Madrid, Spain) Runyon, Noel (Campbell, CA) Schreier, Elliot (National Technology Center, American Foundation for the Blind) Thompson, Wayne (Kentucky Department for the Blind, Technical Services Unit) van den Meiracker, Maud, PhD (COI/ECC, The Netherlands) Vanderheiden, Gregg C., PhD (Trace R&D Center) Access to Graphics-Based Operating Systems for Blind Individuals: +Systems 3+ Model Project Team: Gregg C. Vanderheiden, Ph.D.; Charles C. Lee, M.S.; David. N. Kunz, M.S. Background Graphics-based operating systems pose severe obstacles to computer users with blindness and certain other visual impairments, as quite a bit of the information presented takes forms other than unformatted text. Some of these forms are: operating system constructs such as menus and dialog boxes, text in varying sizes, fonts and attributes, icons or stereotypic images, and diagrammatic or pictographic images. The first challenge in allowing transparent access to such systems for blind users is to locate operating system +access hooks+: points at which screen information can be accessed in order to be sent to some alternative output system such as a braille display or speech synthesizer. The second and equally important challenge, however, is to provide a structured interface for presentation of the screen information so that blind users can interact with the computer in an effective and timely way. Approach The first challenge in providing access (outputting screen information) is being addressed as part of other Trace Center projects, and through cooperative links to other researchers and developers. The second challenge (structuring the user interface) is being studied in terms of a +Systems 3+ model. This model allows for three different approaches to screen access, which could coexist in eventual commercial implementations. In System 1, only a special access program is used for navigating the screen (no additional special hardware component). All commands and requests for information from the user to the access program are issued using the keyboard. In System 2, a touch tablet is used. The user can touch on +speed lists+-areas of the tablet that correspond to menus and messages-in order to specify what information is to be read back. System 2 does not provide any additional access to the computer, but does provide faster access. Neither System 1 nor System 2 can deal directly with graphics information. In System 3, a +virtual tactile tablet+ is added to the system. This tactile tablet allows the user to +feel+ the image on the screen by using a mouse- like puck which contains a vibrating tactile array. Progress A prototype access system is being developed as part of a cooperative effort between the Trace Center and Berkeley Systems, Inc., in conjunction with Apple Computer, Telesensory Systems, Inc., and Articulate Systems. Berkeley Systems, Inc. provides expertise from their work in developing a system for providing voice output access to screen information. Apple Computer provides technical assistance with operating system hooks, TSI with tactile imaging arrays (Optacon II), and Articulate Systems with voice input for the system. The tablet for access in System 2 and System 3 has been completed, along with driver software. Simulations have been run informally with blind users to obtain initial feedback on the operation of the tablet. (These simulations included a hands-on demonstrations at the 1989 National Federation of the Blind conference.) Performance is being refined based on this feedback in preparation for formal experimental testing. Now that the basic hardware has been assembled, the development the operational software and the system interconnections has been initiated. The hardware from these systems is also being used in the Tactile Perception and Business Graphics study series. Publications Vanderheiden, G. C. (in press). Nonvisual alternative display techniques for output from graphics-based computers. Journal of Visual Impairment and Blindness. Vanderheiden, G. C. (1988, Oct.). A multi-sensory, nonvisual interface to computers for blind users. In Proceedings of the Planning Workshop on Access to Computers by Blind Individuals. Madison, WI. Tactile Perception Studies (2-D) and Business Graphics Studies Project Team: Gregg C. Vanderheiden, Ph.D.; Steven Wiker, Ph.D.; David N. Kunz, M.S.; Steven Arndt, M.S. Background A critical component of access to computers for individuals who are blind is the user's ability to deal with information which cannot be easily interpreted and presented in words. Clearly some type of tactile image presentation (such as that discussed for System 3 in the Systems 3 Model) will be necessary. The approach currently being explored at the Trace Center is the use of a vibrating tactile array to present images. It is not clear, however, that this is the best approach for presenting information tactilely. Moreover, there are several aspects of this type of tactile presentation which can be varied, and their relative efficacies are also not known. The purpose of this study series will be to examine several different tactile presentation mechanisms and compare their relative efficacy. Approach In this study series, static raised line drawings, static raised dots, static raised metal pins, and vibrating metal pins will all be compared. Blindfolded, recently blinded, and congenitally blind individuals will be presented with stimuli, and the speed and accuracy of their identification of the stimuli will be noted. The stimuli used in the first series will be simple geometric shapes. The stimuli used in the second series of studies will be simple business graphics such as bar charts, pie charts, and diagrams. In the first study series, the individuals will simply be asked to identify the shape and relative size (big or small) of the object. In the second study series, the individuals will be asked to derive information from the charts, such as which bar is the highest, which bar is the lowest, the height of the highest bar, the number of pieces in the pie chart, etc. Progress Software and studies are currently being designed. Initial studies should be completed during 1990. The second study series is scheduled for 1991. Publications Vanderheiden, G. C. (1988, Oct.). A multi-sensory, nonvisual interface for computers for blind users. In Proceedings of the Planning Workshop on Access to Computers by Blind Individuals. Madison, WI. Graphics Environment Integrated Software Package for Blind Users Project Team: Gregg C. Vanderheiden, Ph.D.; David N. Kunz, M.S.; Sue Melrose, M.A. Background As computer systems are advancing, the trend is toward more and more graphics-based display systems. These new systems are very visually oriented, using graphics, pull down menus, scroll bars, and other visual metaphors. These new graphic systems pose problems for people who are severely visually impaired and blind. The problems posed by these systems can be broken down into two categories: access and efficiency. The access problem arises from the fact that many of these visual metaphors, such as scroll bars, cannot be effectively handled using the same strategies that have been used in the past. As a result, alternative techniques and strategies need to be developed to provide access to these functions. As access to graphic operating systems and programs is developed, the second problem comes into play, that of efficiency. In many cases, the access techniques developed for the blind individual to access the computer are circuitous or based on a reinterpretation of the visual interface into a form which can be perceived by the blind person. A better approach would be an interface which is custom-designed for a person who has no sight. In effect, this would be designing the computer interface which would have been developed first if the majority of computer users were blind. The purpose of this project is to develop an integrated software package (word processing, spread sheet, data base) which is both accessible and efficient for a blind user. It is also desired that this integrated software package be maximally compatible with standard software used by sighted individuals. Approach The approach being taken in this project therefore is to work with a standard integrated software package (Microsoft Works) and adapt it for access by users who are blind. This will be done in a two stage process. First, the general accessibility to the operating system and software will be accomplished through the +Systems 3+ project. This will provide access to the basic operating system constructs used in this graphics-based program. The haptic-tactile interface of +System 3+ will also be used to provide access to the chart- and graph-based portions of the program. Phase II of the project will then involve looking at the specific aspects of the Microsoft Works package which do not lend themselves to effective access and control through operating system constructs. Custom dialogue boxes, tool menus and palettes are examples of features which are specific to programs and not usually available through the operating system constructs. Special adaptations or extensions to the +Systems 3+ access hardware and software will be made in order to transfer these control and feedback structures into a form which is more accessible and efficient for users without sight. The advantage of this two stage approach is, first, that it provides maximum access to other software in addition to the integrated package. Second, it will help us to better determine the types of problems which cannot be addressed purely from an +accessible system+ standpoint and which must be dealt with by the application program manufacturers. Progress Phase I of the project is being carried out in parallel with the +Systems 3+ project. The basic interface being developed will be used both in this program area and in the Systems 3 program area. As discussed earlier, the basic hardware for Version 1 of this system is completed. For phase II of this project, Microsoft Works is undergoing evaluation with the interface system as it is being developed. In addition, training programs for the system (very critical, due to the lack of blind users experienced with graphics-based systems) are being developed. Several audio tape training programs are currently being evaluated. Effect of Control on Auditory Text Presentation Project Team: Jon R. Gunderson, M.S. Background Many research and development efforts directed toward computer access for blind individuals have concentrated on providing access to standard computers and operating systems. However, relatively few have concentrated on isolating the factors involved in optimizing the interface for the blind user. In order to create the most effective access systems, more needs to be understood about how different variables in text presentation interact in actual situations of use. In addition, as the newer systems evolve to include ever-increasing sound capabilities and built-in speech, it will be important to have information as to the appropriate characteristics of a speech synthesizer for use by individuals who are blind. There are many speech synthesizers and +screen reading+ software programs on the market, designed to provide voice- output access to screen information. Many of these systems provide control over variables such as speech rate. However, there are questions that remain unanswered regarding which speech rates are useful for different types of tasks. Are there optimal speech rates for comprehension? How do these rates depend upon the user and their level of experience? How do they relate to the type of text being read and the context? Are variable types of rate control (such as parsing) effective in increasing comprehension? The answers to these questions will enable developers to optimize their hardware and software, and should also encourage the development of types of control that currently do not exist or are too rudimentary. Approach Current literature on comprehension of spoken text and accelerated speech (most of it concerned with high-speed tapes) indicates four main factors in the intelligibility of high-speed speech: 1) Training or familiarity with high- speed speech; 2) prosodic information included-such as parsing of words and stress on content words; 3) speech rate; and 4) length of sentences and average word length. All of these factors are to be considered in this research program. The research is to consider text comprehension in the context of computer operation. Single word comprehension and continuous speech comprehension will be investigated, in the context of prose and also in terms of software menus, error prompts and dialog boxes (messages requiring a response). Sentence comprehension is to be studied in text, but also in status messages (which may appear unexpectedly) and prompts. Reading of longer passages is to be analyzed in terms of context (reading unfamiliar text vs. editing one's own writing). The research will also investigate the potential for future systems in which audio information is used in its own optimal form, rather than as an analog to a visual display. The results of this research will be useful in the future design of speech synthesizers, especially in algorithms for acceleration. They will also have implications in the design of voice-based computer access systems. Finally, they will serve to test theories of speech perception, and apply them to synthetic speech. Progress A review of current literature and research has been carried out. Initial experiments on speech comprehension have been designed as well. Four experiments are proposed. Two of the experiments will use standardized tests of speech perception and two will use new tests based on multiple syllable words. The tests will include the individual words out of context (emulating menu selection) as well as sentences which are semantically meaningful. Three theories of speech perception will be tested in the design of the accelerated speech materials. For the studies two groups of subjects will be used: one without previous exposure to synthesized speech and one that has had experience with it. This will allow the effect of training and experience to be evaluated. Publications Gunderson, Jon. (1988, Oct.). Information processing model of human computer interaction for people with blindness and severe visual impairments. Proceedings of the Planning Workshop on Access to Computers by Blind Individuals. Madison, WI. Auditory Redundancy for Hearing Impaired Individuals Project Team: Gregg C. Vanderheiden, Ph.D.; Charles C. Lee, M.S. Background Currently, people with hearing impairments have little or no difficulties in using computers. The use of sound as a standard feature has been minimal, usually no more than a +beep.+ If the beep is accompanied by some visual event, no problem is encountered. If the beep is not accompanied by a visual event, the deaf person may miss the cue, may deduce what change or error has occurred, or may use a simple sound alert device with a flashing light to detect the sound. However, the increasing sophistication of synthetic and digital speech technologies has made it easier and more desirable for computer companies to consider incorporating voice output into their products. This would probably take the form of standard voice features in the operating system (such as voice output of error messages) or available voice features that could be +called+ from the operating system by application programs. Approach As a first step in addressing these problems, a proposal has been made for the incorporation of a +hearing impairment flag+ in standard computer operating systems. Such a flag would appear along with other control settings for the operating system. The flag would provide a means for the user to signal cooperating software and operating systems that the user cannot hear any sounds emitted by the computer. Programs and operating systems could then accompany any beeps with some type of visual event on the computer screen. Simple beeps might correspond to a flashing of the menu bar or screen border. More complex tonal output could be presented to the user in some type of graphic that would appear on the screen. The hearing impairment flag also presents the possibility of +closed captioning+ for computer programs. This could work in two ways. In the first method, cooperating programs that used speech output could check the hearing impairment flag through the operating system. If the flag were set, then the program could display a caption along with any voice output. The second closed captioning method will be possible once computers have standard text-to-speech capabilities (in which case application programs can simply send a string of text to the operating system in order to have it spoken). In this scenario, an application program would simply tell the operating system to say something; the operating system itself would then check the hearing impairment flag and automatically provide closed captioning. This would eliminate the need for programs to be individually equipped for closed captioning (but would not preclude this option). Progress The proposal for a hearing impairment flag is being fed to computer companies which are pursuing specific disability development efforts. It is also being incorporated into the +Considerations+ document of the Industry/Government Cooperative Initiative on Computer Accessibility, and through that channel distributed to any other interested companies, organizations or individuals. The implementation of the recommendations could be carried out in stages-in fact they would need to be, since later stages require built-in text-to-speech capabilities which are not yet a standard part of any computer operating system. The stages for implementation would be: 1) Inclusion of a hearing impairment flag in the control settings of the operating system; 2) Implementation of visual events to correspond to beeps triggered by the operating system; 3) Provision of closed captions for any voice or complex sound events necessary for use of the operating system; 4) Provision of closed captioning tools for use by third party developers; 5) Provision of auto-captioning capability. In addition, application programs can begin to check for the hearing impairment flag and provide visual cues to any auditory events. The application programs most likely to make use of such a flag would be programs designed for education or specifically for the disability field. However, government legislation regarding computer access may encourage the use of the hearing impairment flag by business software vendors as they incorporate sound into their products.