bgrahame@cix (Bob Grahame) (10/23/90)
It seems to me that VR so far is 95% Visual 5% Tactile. While this may be fine for the majority, I foresee an increased sense of isolation for the visually handicapped, who will be left alone as the rest of us stride off into a virtual future. On the other hand, VR would be of great interest to the physically handicapped, who would be able to 'compete' on more equal terms in a non-physical universe. I think therefore that there needs to be more thought about non-visual VR sensoriums here at the beginning of the technology. Your views, Ladies and Gentlemen ? Bob.
esdvt@esdvt.esd.sgi.com (ESD DVT) (10/25/90)
Why wouldn't people who are visually handicapped just have the dead nerves or whatever was failing bypassed by hardware so they could "see" I think that as stuff like socketing in prosthetics becomes more viable this sort of thing would be possible? any flames/comments/support? Ben Pappas pigboy%esdvt.esd@sgi.com I have no opinions, neither does Silicon Graphics
pepke@SCRI1.SCRI.FSU.EDU (Eric Pepke) (10/26/90)
In article <9961@milton.u.washington.edu> esdvt@esdvt.esd.sgi.com (ESD DVT) writes: > Why wouldn't people who are visually handicapped just have the > dead nerves or whatever was failing bypassed by hardware so they > could "see" There has been some work along these lines, but there are some difficulties. One is that vision requires come brain development which can only occur at an early age and can only occur in response to visual stimuli. There have been cases where people blind from birth were given operations to give them sight, but they lacked the cognitive facilities to make sense out of the data, which came across as distracting noise. For others, bypasses should eventually be possible, but I think that by then one of two things (or both) will be true: implantable prostheses will make these people no longer blind, or the neural connections will be the stuff of virtual reality. The former is more likely; the latter is more romantic. However, I don't think the idea that VR must neccessarily exclude the disabled (at least any more than normal reality does) is accurate. The VPL setup has a pair of stereo headphones and a processor which models the filtering of the ear and head to do binaural sound. It's still quit primitive (less so than the graphics, though) but it is amazingly effective. Anybody who has heard a good binaural recording, such as in the sound booths at Disney/MGM studios, knows that it can convey a tremendous amount of spatial information. A good field of research might be the investigation of audio "glyphs." A glyph is a graphical object which simultaneously conveys more than one piece of information. An audio glyph might be a burst of sound which does the same thing. Audio glyphs might be realistic or purely symbolic. For example, imagine a virtual reality of a library of world history. The WWI section might be a burst of a scratchy recording of "Over There." The WWII section might be a couple of clunks of marching feet from some Nazi propaganda film. Vietnam would be the sound of helicopters. Asian, African, European history and subsections thereof might be snatches of characteristic music from the areas. (Nostalgic note--I once wrote a deranged little game for the TRS-80 called "Find the Plutonium." There was nothing on the screen at all, just an occasional ticking from the "geiger counter." Based on the frequency of the ticking, you had to find your way to the plutonium.) Eric Pepke INTERNET: pepke@gw.scri.fsu.edu Supercomputer Computations Research Institute MFENET: pepke@fsu Florida State University SPAN: scri::pepke Tallahassee, FL 32306-4052 BITNET: pepke@fsu Disclaimer: My employers seldom even LISTEN to my opinions. Meta-disclaimer: Any society that needs disclaimers has too many lawyers.
pathak@mbunix.mitre.org (Pathak) (10/27/90)
I remember reading in the Chicago Tribune about 6 months ago about a new experimental technique that allows human nerves to be joined with computer wafers. It seems that the wafers have holes in them and they can get the nerves to tie into the receptacles. The article stated that the researchers hope perfect this technique and then go on to develop a whole new generation of prosthesis (sp?) devices that are smart enough to react the information coming from the nerves. I believe they were talking about such devices for accident victims. Heeren Pathak pathak@mitre.org #include <standard.disclaimer>
hlr@uunet.UU.NET (Howard Rheingold) (10/27/90)
bgrahame@cix (Bob Grahame) writes: >It seems to me that VR so far is 95% Visual 5% Tactile. Very good work on tactile and haptic elements of VR is being conducted at the University of North Carolina, at the Media Lab, and at ACROE in Grenoble, France. Also some work in England. Frederick Brooks and his associates at UNC, Margaret Minsky at Media Lab, Annie Luciani and Claude Cadoz and Jean-Loup Florens in Grenoble. Japanese researchers at MITI and ATR are also interested, although I did not see any of their prototypes in this direction. The definitive article about the use of haptic feedback in scientific visualization was Frederick Brooks' presentation at SIGGRAPH. It is in the 1990 Siggraph proceedings, available from the ACM.
sobiloff@acc.stolaf.edu (Chrome Cboy) (10/28/90)
In article <9961@milton.u.washington.edu> esdvt@esdvt.esd.sgi.com (ESD DVT) writ es: > Why wouldn't people who are visually handicapped just have the > dead nerves or whatever was failing bypassed by hardware so they >could "see" I think that as stuff like socketing in prosthetics becomes more >viable this sort of thing would be possible? any flames/comments/support? [Note to the moderator: I'm assuming that this post passes the non-cyberpunk kosherness test since it was posted. Please feel free to put my reply in the bit bucket if my assumption is incorrect. (A receipt would be appreciated, however... :-) ] Something like this certainly would be nice for both visually handicapped individuals and non-handicapped individuals. However, our current level of technology, not to mention our very limited knowledge of higher-order visual processing, currently prohibits anything like this. Another problem is that, even if we were able to perform such operations, they would not help individuals who had been blind since birth or at an early age. There are numerous physiological developments in the visual centers of the brain that occur during the first year or so of life. If there is no vision during this period these developments will not occur, thus rendering the individual physiologically incapable of processing visual information, even if the faulty receptors (eyes) are replaced with functional units. If anyone is interested in this area, a good place to look for information is a college-level Sensation & Perception textbook. Also, if you want pointers to papers, these are some good ones to look up: Held, R. (1965) Plasticity in sensory-motor systems. "Scientific American," 213, 84-94. Regal, D.M., Boothe, R., Teller, D.Y., & Sackett, G.B. (1976) Visual acuity and visual responsiveness in dark-reared monkeys. ("Macaca nemestrina"). "Vision Research," 16, 523-530. Hubel, D.H., & Wiesel, T.N. (1970a). The period of susceptibility to the physiological effects of unilateral eye closure in kittens. "Journal of Physiology," 206, 419-436. -- ______________ _______________________________________________________/ Chrome C'Boy \_________ | "One of the biggest obstacles to the future of computing is C. C is the last | | attempt of the high priesthood to control the computing business. It's like | | the scribe and the Pharisees who did not want the masses to learn how to | | read and write." -Jerry Pournelle |
mark@cis.ohio-state.edu (Mark Jansen) (10/30/90)
Does anyone know where to start to learn about computers and the
handicapped in general as well as the more specialized virtual
reality issues.
Seems to me that there is a IEEE Sig that is dedicated to
handicap stuff. Is it any good?
As always is there an to date, textbook on the subject of
handicaps and computing. For example, what are numercially
the most important handicaps in our society. What percentage
blind, deaf, mute, paralyzed etc?
--
Mark Jansen, Department of Computer and Information Science
The Ohio State University; 2036 Neil Ave., Columbus, OH USA 43210-1277
mark@cis.ohio-state.edumark@cis.ohio-state.edu (Mark Jansen) (10/30/90)
In article <10037@milton.u.washington.edu> pathak@mbunix.mitre.org (Pathak) writ es: > >I remember reading in the Chicago Tribune about 6 months ago about a new >experimental technique that allows human nerves to be joined with >computer wafers. It seems that the wafers have holes in them and they can >get the nerves to tie into the receptacles. The article stated that the >researchers hope perfect this technique and then go on to develop a whole >new generation of prosthesis (sp?) devices that are smart enough to react >the information coming from the nerves. I believe they were talking about >such devices for accident victims. > I seem to remember that if you take a suspension of nerve cells you can get them to settle and attach to a integrated circuit kind of wafer and then do some training. What appears to be new then would be how to get nerve cells of a living human being to proliferate and grow out. Is that really posible yet? -- Mark Jansen, Department of Computer and Information Science The Ohio State University; 2036 Neil Ave., Columbus, OH USA 43210-1277 mark@cis.ohio-state.edu
lishka@uwslh.slh.wisc.edu (a.k.a. Chri) (11/01/90)
sobiloff@acc.stolaf.edu (Chrome Cboy) writes: >In article <9961@milton.u.washington.edu> esdvt@esdvt.esd.sgi.com (ESD DVT) wri t >es: >> Why wouldn't people who are visually handicapped just have the >> dead nerves or whatever was failing bypassed by hardware so they >>could "see" I think that as stuff like socketing in prosthetics becomes more >>viable this sort of thing would be possible? any flames/comments/support? >Something like this certainly would be nice for both visually handicapped >individuals and non-handicapped individuals. However, our current level of >technology, not to mention our very limited knowledge of higher-order visual >processing, currently prohibits anything like this. Well, "yes and no" to the above statement. I saw a remarkable show a few years back that dealt with computers and handicapped people. Possibly the most amazing development that was presented was artificial vision. It has been a while since I have seen this program (so I don't remember the name of it, or that many details), but I am nearly positive it was on PBS. The artificial vision research was being done with a person who was completely blind. Note that he was not blind from birth, but rather it was caused be some sort of accident or disease later in life. The important thing to remember is that this person *had* be able to see earlier in his lifetime. The experiment that was being carried out was roughly as follows. The scientist had a computer hooked up to a camera, which translated images into a small array of "block" patterns. The "blocks" in this case were digital; in other words, they were either lit or dark. Think of the array as a small monochrome screen. [The computer was large by current standards, but I assume it could be shrunk to a smaller size today.] The computer was connected to the blind person via wires to some part of the person. I don't remember exactly *how* it was being done; possibly the wires led to some unused vision center. I do distinctly remember that the patient was "seeing" bright lights for each "block" the computer turned on, and that this "vision" was being approximated using what remained of the patients functional vision organs. What was happening was that the wires that were connected to the patient were duplicating the visual array created by the computer, so that the blind person could "see" a crude (by normal vision standards) image of what was in front of him. [Note: another way to do this is through a "pain array"; see the note at the end of this posting for more info on this.] There were certainly limitations to this system. It was very low resolution, it was monochrome, and it was slow. Furthermore, only a single image was being presented, so there was no depth information available (as would would get from stereo images). It was also (at the time) very "nonportable," because it is hard to lug a computer the size of a VAX around with you. However, at the time the show was produced the blind patient was able to make out objects in front of him. In fact, he was able to do so well enough to walk amongst them. This system likely would not work in a complex environment (e.g. a street corner with different colored lights for stop, caution, go, walk, don't-walk), but it was an important first step. It was very touching to see the look on the patient's face when first seeing images after a long period of blindness. Unfortunately, I don't have any of the details on this. If anybody knows more about this research, please post more information here. This is certainly relevant to virtual reality because it presents the beginnings of a way to "jack into" a cyberspace. One final note: the "pain array" I referred to above is another form of artificial "vision." The system basically works the same way as the above experiment, except the vision matrix that the computer builds is mapped onto an area of the patients skin using tactile response. The form I have heard of is a simple two-dimensional array of semi-blunt needles. In this case the bright "blocks" in the computers vision matrix are stimulated by pressing the appropriate needles against the skin of the patient (triggering pain sensors), while the "dark" blocks are areas of no stimulation. The "pain array" (which is my term) can be set up over a large, seldom used area (e.g. the pain sensors on a persons back) or over a smaller area (e.g. the inside of a person's forearm). I am not sure how well these "pain arrays" work. >Another problem is that, even if we were able to perform such operations, >they would not help individuals who had been blind since birth or at an >early age. There are numerous physiological developments in the visual >centers of the brain that occur during the first year or so of life. If there >is no vision during this period these developments will not occur, thus >rendering the individual physiologically incapable of processing visual >information, even if the faulty receptors (eyes) are replaced with functional >units. Most definitely! The above research worked because the blind patient *had* seen in the past, and knew what to expect and what the shapes meant. Using the same system for a person blind from birth would (at the very least) require that the patient be taught what the foreign images are; i.e. how they map to real-world objects. -- Christopher Lishka 608-262-4485 "Dad, don't give in to mob mentality!" Wisconsin State Lab. of Hygiene -- Bart Simpson lishka@uwslh.slh.wisc.edu "I'm not, Son. I'm jumping on the bandwagon." uunet!uwvax!uwslh!lishka -- Homer Simpson
warren@debra.doc.ca (Warren Baird) (11/03/90)
In article <10438@milton.u.washington.edu> lishka@uwslh.slh.wisc.edu (a.k.a. Chr i) writes: > >The computer was connected to the blind person via wires to some part >of the person. I don't remember exactly *how* it was being done; >possibly the wires led to some unused vision center. I did some research in the area of artifical eyesight for a High School report, and I seem to remember that a similar effect was achieved by attaching a teflon plate to the visual cortex along the back of the brain. Electrodes embedded in the teflon could be switched on and off to create points of light in the person's visual field. >There were certainly limitations to this system. It was very low >resolution, it was monochrome, and it was slow. I seem to recall (this was a few years ago) that they expected to be able to achieve resolutions similar to the resolution of a scoreboard at a sports stadium. >-- >Christopher Lishka 608-262-4485 "Dad, don't give in to mob mentality!" >Wisconsin State Lab. of Hygiene -- Bart Simpson > lishka@uwslh.slh.wisc.edu "I'm not, Son. I'm jumping on the bandwagon." > uunet!uwvax!uwslh!lishka -- Homer Simpson -- Warren Baird | warren@dgbt.doc.ca ...utzoo!dciem!nrcaer!dgbt!warren Doing a Co-op term at Communications Canada, Ottawa
sobiloff@acc.stolaf.edu (Chrome Cboy) (11/04/90)
In article <10438@milton.u.washington.edu> lishka@uwslh.slh.wisc.edu (a.k.a. Chr i) writes: >sobiloff@acc.stolaf.edu (Chrome Cboy) writes: >>In article <9961@milton.u.washington.edu> esdvt@esdvt.esd.sgi.com (ESD DVT) wr i >t >>es: >>> Why wouldn't people who are visually handicapped just have the >>> dead nerves or whatever was failing bypassed by hardware so they >>>could "see" I think that as stuff like socketing in prosthetics becomes more >>>viable this sort of thing would be possible? any flames/comments/support? > >>Something like this certainly would be nice for both visually handicapped >>individuals and non-handicapped individuals. However, our current level of >>technology, not to mention our very limited knowledge of higher-order visual >>processing, currently prohibits anything like this. > >Well, "yes and no" to the above statement. I saw a remarkable show a >few years back that dealt with computers and handicapped people. >Possibly the most amazing development that was presented was >artificial vision. It has been a while since I have seen this program >(so I don't remember the name of it, or that many details), but I am >nearly positive it was on PBS. I saw this segment as well; I think it was a Nova special, if I recall correctly. The segment hightlighted the work being done by William Dobelle, a summary of who's work is included at the end of this post. The military was also doing basic research along these lines in the late 60's, early 70's, but with different goals in mind. (I think they were looking to actually "jack in" a fighter pilot, bypassing controls such as the stick and throttle.) Their stumbling block was that the areas in which the electrode array were implanted would become physically damaged due to the current being used, rendering the area unusuable after only a couple of months. >One final note: the "pain array" I referred to above is another form >of artificial "vision." The system basically works the same way as the >[...] >forearm). I am not sure how well these "pain arrays" work. They work remarkably well once the patient has become accustomed to the system. The largest drawback has been trying to miniaturize the setup-- it is currently still a large chair that one must sit in. What with the advances in materials science and the emerging ability to control individual atoms in a manufacturing process I don't see why a practical, mineaturized system won't be produced in a decade or so. (Assuming nothing better comes along in the meantime, which is an admittedly large assumption. :-) Anyway, the article I mentioned previously: [Taken from "Sensation and Perception", 3rd Ed., E. Bruce Goldstein, Wadsworth, 1989, 50-51.] William Dobelle and a team of researchers have developed plans for a visual prosthesis: an artificial eye that can produce vision in the blind by sending electrical signals to the visual cortex. One proposed system, shown in the figure on the left, consists of a subminiature television camera mounted in a glass eye. The TV camera picks up an image from the environment and converts it into a pattern of electrical signals that is transmitted to a tiny computer mounted in a pair of dummy glasses. After being processed by the computer, these signals stimulate an array of electrodes placed on the surface of the visual cortex, causing the person to see an image corresponding to the original image received by the TV camera. Although the system shown in the figure on the left does not yet exist, some totally blind volunteers have had an array of 64 electrodes implanted on their visual cortex. A drawing of this array, based on an X-ray, is shown on the right. Wires from the electrodes pass through a small opening in the skull, at the back of the head, and then wind beneath the scalp in an S-shaped loop, before coming through the skin and terminating in a connector attached to the skull with special bone screws. Initial testing indicates that stimulation of the cortex by a single electrode causes the observer to perceive a phosphene--a small glowing spot of light that appears to be located at about arm's length. Stimulation of a number of electrodes causes a number of phosphenes to be seen as a pattern. For example, Craig, a 35- year-old government employee blinded 15 years earlier in an automobile accident, was able to see letters, geometrical shapes, and simple pattern in response to various patterns of stimulation of his electrode array. When a TV camera was connected to the system, Craig could also detect white horizontal and vertical lines on a dark background. These initial results are promising, although many problems must be solved before a visual prosthesis becomes available. For example, phosphenes are not always seen in the position corresponding to the position of the electrode on the cortex. Adjacent electrodes may produce phosphenes that are spaced far apart in the visual field, and adjacent phosphenes are sometimes produced by widely spaced electrodes. Additionally, some phosphenes are too bright and obscure others, and phosphenes often flicker, even in response to continuous electrical stimulation. The next step is the development of a 512-electrode stimulation system that would transmit complicated information rapidly enough so that the prosthesis could be used by a person as he or she walks through the environment. (Dobelle et al., 1974, 1976; Dobelle, 1977). -- ______________ _______________________________________________________/ Chrome C'Boy \_________ | "One of the biggest obstacles to the future of computing is C. C is the last | | attempt of the high priesthood to control the computing business. It's like | | the scribe and the Pharisees who did not want the masses to learn how to | | read and write." -Jerry Pournelle |
fenn@wpi.wpi.edu (Brian Fennell) (11/04/90)
In article <10438@milton.u.washington.edu> lishka@uwslh.slh.wisc.edu (a.k.a. Chri) writes: ... >I saw a remarkable show a >few years back that dealt with computers and handicapped people. >Possibly the most amazing development that was presented was >artificial vision. It has been a while since I have seen this program >(so I don't remember the name of it, or that many details), but I am >nearly positive it was on PBS. > >The artificial vision research was being done with a person who was >completely blind. Note that he was not blind from birth, but rather >it was caused be some sort of accident or disease later in life. The >important thing to remember is that this person *had* be able to see >earlier in his lifetime. > >The experiment that was being carried out was roughly as follows. The >scientist had a computer hooked up to a camera, which translated >images into a small array of "block" patterns. The "blocks" in this >case were digital; in other words, they were either lit or dark. >Think of the array as a small monochrome screen. [The computer was >large by current standards, but I assume it could be shrunk to a >smaller size today.] > >The computer was connected to the blind person via wires to some part >of the person. I don't remember exactly *how* it was being done; >possibly the wires led to some unused vision center. I do distinctly >remember that the patient was "seeing" bright lights for each "block" >the computer turned on, and that this "vision" was being approximated >using what remained of the patients functional vision organs. What >was happening was that the wires that were connected to the patient >were duplicating the visual array created by the computer, so that the >blind person could "see" a crude (by normal vision standards) image of >what was in front of him. [Note: another way to do this is through a >"pain array"; see the note at the end of this posting for more info on >this.] Your memorys are a bit inaccurate (if I am remembering the same article): The article, I beleive, was on the series called "the Brain" on PBS. (but I am not sure.) There were two experiments, one with a 2x6 surgical inplant that sent braile images, one that used physical stimulus of the skin of the back (vibrating, blunt, wooden or metal pegs aproximatly 20x20, the vibration caused the physical response, no "pain" involved). The man in the second experiment WAS blind from birth, he had never seen the flame on a candle before. (to a blindman flame has no shape just heat) I remember that the aparatus of the second experiment was the peg-array afixed to a chair, and a hand held, video camera, attached by wires to the chair and a wall monitor of some sort (of the same resolution). I don't remember him walking around with this device. Additional information: The picture was being processed by the same part of the brain as eye images, despite not using the optic nerve (in fact using the back). This I think was determined using CAT-scan, and radio-active gluecose. Asside from my own personal experience: There exists a device called an "Opticon" that uses tiny vibrating metal pegs to stimulate an image on the index finger, and a small one character scanner that can be dragged along a line of text. I have tried it, no pain. Anyone can use it with practice, although people who are blind from birth need to be taught what a printed character looks like. These devices are relatively common in the blind community. This idea of people who are blind from birth not being able to use the visual center of the brain is 100% wrong. Brian Fennell=fenn@wpi.wpi.edu
lishka@uwslh.slh.wisc.edu (a.k.a. Chri) (11/05/90)
[Chris gave me permission to post the following email between us.
Feel free to comment on our discussion. -- Bob Jacobson, Moderator]
(From Bob:)
Especially needed are insights from people in the biomedical
professions. I would especially like to here from someone
in your situation about virtual interface technology's use in
the laboratory, for the benefit of designers who may be thinking
up technology for the health laboratory market.
Hmmm. I read your email message when it first arrived, and have been
thinking about this issue. Unfortunately, I don't yet see a real use
for V.R. interface technology in the lab.
The basic problem is that the health sciences (at least the ones I
have been exposed to) are somewhat behind in computer technology. In
the public health lab that I work for we use a VAX 11/750 (!) as our
main computer, and are trying to migrate to a VAXstation 3100. We
also have minicomputers like Hewlett Packard (3000, I think) and a
Perkin Elmer running a real-time operating system for real-time data
aquisition. We are currently in the process of sending out an RFP for
a new main computer (our old 11/750 is incredibly overloaded) with
some commercial health-lab software.
The health science computer industry seems to lag well behind "cutting
edge" computer technology. It still seems to be buried in programs
that do administrative tasks, maintain databases, and converse with
intelligent lab instruments. Some of the more "exotic" technology we
are looking at is in areas like speech recognition (so pathologists
need not look up from microscopes to log results into a computer) and
electronic report-form editing and preview (to save paper and time).
This, of course, is old-tech in terms of current computer science.
As for V.R. in health sciences, the only thing that I think might help
is scientific visualization software. But even this is a ways off.
I will keep thinking about V.R. in the health sciences. It is an
intriguing question/problem. If I come up with anything, I will
certainly post my thoughts.
.oO Chris Oo.
Christopher Lishka 608-262-4485 "Dad, don't give in to mob mentality!"
Wisconsin State Lab. of Hygiene -- Bart Simpson
lishka@uwslh.slh.wisc.edu "I'm not, Son. I'm jumping on the bandwagon."
uunet!uwvax!uwslh!lishka -- Homer Simpson