zap@lage.lysator.liu.se (H}kan Andersson) (09/14/90)
Some Comments on Virtual Realities, computers
networking and related topics.
by
Haakan 'Zap' Andersson
References: Lot's a chat going on in sci.virtual-worlds.
I have been reading sci.virtual-worlds for quite some time without
saying a lot about it. A lot if things pop into my mind, though, and I
think that I should share my ideas.
First, let me get som definitions right (The way I use them, not
nessesarily the way YOU use them):
A virtual-REALITY is a computer simulation of things in the real world,
desks, lamps, documents to read, sounds to be heard, e.t.c. e.t.c, whereas
CYBERSPACE in my mind is a 'place' where 'data is depicted in it's "Raw"
form', i.e. a space where you navigate in, around, with and about data.
Of course, a VR could be (and is, in a sense) one piece of data in cyber-
space. Some important distinctions:
A VR is three-dimensional, you can first of all see and hear it (that we
have for sure with todays technology, even if the images might be crude),
touch (this exists today in a limited way), smell, taste, and perhaps feel
via ESP, i.e. neural signals fed into the brain with no relation to any
of the five senses. (Forgive me for calling it ESP, but isn't it just that?))
VR is populated with objects that have spatial extents, that you can interact
with in a way similar to the RW (Real World). It may, of course, have abstract
things in it, like people with three legs or talking applepies, but then we
are approaching a virtual-WORLD instead of a virtual-REALITY. A game,
f'rinstance, where you slay dragons, seduce beautiful talking applepies,
and eat lovely young princesses, is a virtual-WORLD, and not a virtual
REALITY. (At least I have never seduced a talking applepie)
CYBERSPACE, on the other hand, is not limited to a finite number of
dimensions, must not nessesarily be populated with objects that have
any meaningful extents in any given directions, it is simply a novel
way (and hopefully useful) of representing data, interacting with it
and navigating through it. Of course, to our mind, and todays technology,
a three-dimensional cyberspace with data represented as objects similar
to VR above is easier to implement. But as the direct hardware connection
to the brain evolves (if it will ever) then these limits can, and will be,
lifted.
Bearing these 'definitions' in mind for VR vs. CYBERSPACE, we may continue
with this tiny thread of unsorted thoughts....
** The 'protocol' that has been mentioned and partially designed in this
newsgroup is interesting, but I see some limitations.... (Yes yes, it's
a pre-pre-pre-before-I-though-t-at-all-draft, I know) and want to throw
in my $0.02. F'rinstance, why limit the placement of an object to three
dimensions? Who said cyberspace isn't 7-dimensioned? Or 37.3 dimensions?
Supply a coordinate count and a number of coordinates. For now, and in the
forseeable future, the count will be 3 and the coordinates will be X, Y and
Z, but you wanted extensibility..... As for the appearance description of
an object, i.e. how-the-darn-thing-looks, then why reinvent the wheel (this
is done too often in computer science) when this already exists: RenderMan.
The only drawback in using renderman is the little line in the RenderMan
spec that says that you must respecify the entire scene for each frame. To
reduce load on the net, we need an extended version of renderman, where you
only respecify those objects you really need to. But since this is into the
domain of fiddling-with-renderman, then it's really up to PIXAR to do this,
since it's 'their' standard. (Hi Steve Upstill@pixar.com :-)
** When you think of it, cyberspace is only way of integrating everything
we ever knew about computer inte something within one single (cyber)inter-
face. Someone pointed out the similarity between navigating in cyberspace
and navigation in a hypertext system. But have anyone thought of the simi-
larity between cyberspace and the GUI's of modern computers?
In a sense, cyberspace is already invented, but today it's only two-
dimensional, it sits on the screen in front of you. Isn't the mouse just a
forerunner of the dataglove? Arn't we already manipulating objects in a
virtual world, when we drag the 'Graphic Examples' drawer in to the
'Trashcan', close the 'Graphics Disc' window, dragging the 'Graphics Disc'
icon over the 'Blank Disc' icon, thus producing a copy?
In a GUI, the programs (tools) and data (projects) are displayed as objects,
and they are in a limited sense objectoriented, i.e. double-clicking on a
data-object (perhaps a DeluxePaint image) will trigger the action associated
with that type of object (Running DeluxePaint). But in a GUI it is rather
limited today. How many times have I not been irritadet when I simply wanted
to LOOK on a DeluxePaint image, and doubleclicking the durn thing loads the
entire 700k worth of program, wasting my time and memory.
The problem with the GUI is that there is only ONE
'default action' for an object. In cyberspace we will need to have any
number of 'actions', but we need some that are 'standard'. F'rinstance, we
may want to EXAMINE the object (scan it just to see what it is), READ the
object (actually plunge into the data it represents), OPERATE the object
(similar to what the GUI does, kick the objects 'default action' to life),
and EDIT the object (start som kind of process that can modify the selected
object). Many more are concievable, but a cyberspace object must at least
have these basic capabilites.
** Another factor is the 'level-of-detail' dimension. In computer gfx, a
constant problem is at what level of detail you should represent an object.
Renderman (mentioned earlier) has some slick solutions to this problem. But
in cyberspace, level-of-detail must be under user control. Perhaps his
eyephones react to slowly because he has a low-end cyber-workstation or
the net is hung with four others having cybersex with a talking applepie,
or whatever. He may want to pull down the level of detail in his surround-
ings, removing the details he do not need right now.
Another use for this 'level-of-detail' is like an extra 'dimension',
especially useful when examining data, Fr'instance, at the 'lowest resolution',
some chunk of data might just be a blob, a pixel, a tone, a smell of burnt
rubber or an image of a talking applepie. At a higher 'resolution', higher
level-of-detail, the same object might have a 'shape' (or smell, flavor or
spice) revealing it's type, and perhaps it's name. This is the level that
most GUI's work at.
Next level down is where the contents of the data is shown in a
summary format, so you 'get the hang of' what it's all about. Next step
is when all of the data is readable. Next step down is when not only the
data, but technical info, like sources, references, storage media info and
such are shown. Further steps may exist, and intermediate steps between the
ones listed may also exist. The user must also be able to 'enlarge' a piece
of the world with a 'cyber-looking-glass', i.e. when he get's to the word
'Hydrogen' in a text, he may use the 'looking-glass' to 'magnify' that word
into higher levels of detail, where you may find things like the inventor
of Hydrogen, it's atomic wheight, the price on the Hydrogen stock-exchange
and other interesting facts about Hydrogen. This is really where hypertext
and cyberspace get to interact.
** It's getting late, and I have to pay me some bills. I hope this little
chat has triggered some thoughts in this nowsgroup. Thank you for your time!
(P.S. I just thought i saw something on the screen....it looked like....
an applepie???)
+ + + + + + + + + + + + +
+ My signature is +
+ smaller than yours! +
+ - zap@lysator.liu.se +
+ + + + + + + + + + + + +broehl@watserv1.waterloo.edu (Bernie Roehl) (09/15/90)
In article <7511@milton.u.washington.edu> zap@lage.lysator.liu.se (H}kan Anderss on) writes: >** The 'protocol' that has been mentioned and partially designed in this >newsgroup is interesting, but I see some limitations.... > F'rinstance, why limit the placement of an object to three >dimensions? Who said cyberspace isn't 7-dimensioned? Or 37.3 dimensions? >Supply a coordinate count and a number of coordinates. For now, and in the >forseeable future, the count will be 3 and the coordinates will be X, Y and >Z, but you wanted extensibility... The idea of "attributes" is nice because it *is* extensible... we specify at attribute as "position" and have its value be x, y, z. Once we figure out a way of handling more dimensions, we can "expand" the object by giving it additional attributes for the "extended" dimensions (say, an attribute called EXTENDED_POSITION that, as you suggest, has a coordinate count and an essentially unlimited number of coordinates). Objects that have POSITION but lack EXTENDED_POSITION are simply "flat" in the unused dimensions. >..... As for the appearance description of >an object, i.e. how-the-darn-thing-looks, then why reinvent the wheel >when this already exists: RenderMan. I already suggested RenderMan in an earlier article; however, the RenderMan appearance of an object is only *one* representation, and simpler (wire-frame or whatever) versions should be present as well. Small computers may not implement even a subset of RenderMan, but still want to know the basic size and shape of an object. Some objects may not even have a visual appearance at all, but exist on some other sensory level entirely. >Fr'instance, at the 'lowest resolution', >some chunk of data might just be a blob, a pixel, a tone, a smell of burnt >rubber or an image of a talking applepie. At a higher 'resolution', higher >level-of-detail, the same object might have a 'shape' (or smell, flavor or >spice)... Right -- exactly! >The only drawback in using renderman is the little line in the RenderMan >spec that says that you must respecify the entire scene for each frame. To >reduce load on the net, we need an extended version of renderman, where you >only respecify those objects you really need to. Since the frame rendering is done on the local system, not on the host, this isn't an issue; you keep track of each object's RenderMan specification, and alter their position/orientation whenever the room tells you that the object has moved. (This brings up a point I forgot to mention earlier; the building is responsible for event synchronization, so that I move at the same "time" from the standpoint of every occupant of the room, and the "swooshing" sound I make is co-ordinated with my movement). >But have anyone thought of the similarity between cyberspace and >the GUI's of modern computers? Yes. (At least, *I* have -- and I assume others as well). >In a sense, cyberspace is already invented, but today it's only two- >dimensional, it sits on the screen in front of you. Right. The extension to three dimensions is a critical element, as is the notion of "sharing" that world with other people. -- Bernie Roehl, University of Waterloo Electrical Engineering Dept Mail: broehl@watserv1.waterloo.edu OR broehl@watserv1.UWaterloo.ca BangPath: {allegra,decvax,utzoo,clyde}!watmath!watserv1!broehl Voice: (519) 885-1211 x 2607 [work]
ins_atge@jhunix.HCF.JHU.EDU (Thomas G Edwards) (09/15/90)
In article <7511@milton.u.washington.edu> zap@lage.lysator.liu.se (H}kan Anderss on) writes: >** Another factor is the 'level-of-detail' dimension. In computer gfx, a >constant problem is at what level of detail you should represent an object. One of the best ways to fight the bandwidth/resolution issue is to examine what human senses do. We have the same problem that VR representations do...there is just too much bandwidth of senses. There is no way we could process all that information without attentional subsystems. Basically, if something doesn't have our "attention" we won't respond to it. Anyone who has slept through rain knows that it is often easier to sleep through continual white noise than someone making noise in the kitchen (even at smaller volumes). White noise tires out the attentional system, since the noise is more or less continuous. Someone in the kitchen might open a refrigerator, bang a plate, do some mixing, etc. Everytime the noise changes its character, you notice it. Eye has a built-in attentional system. It only has significant resolution for areas it is directly pointing in. The resolution falls off greatly in peripheral vision. If something catches your eye in peripheral vision, you must move your eyeball to point at it to bring the image to the fovea before you can really see it well. Theoretically, we can get away with only presenting sensory information which brain will be attentive to. Eyes are a little bit difficult, but I could imagine a laser eye position sensor seeing where it is looking, and only rendering in high-resolution areas of a screen which show up in the foveal region. -Thomas
brucec%phoebus.phoebus.labs.tek.com@RELAY.CS. (Bruce Cohen;;50-662;LP=A;) (09/17/90)
In article <7624@milton.u.washington.edu> watserv1.waterloo.edu!broehl@ria.ccs.u wo.ca (Bernie Roehl) writes: > > > > In article <7511@milton.u.washington.edu> zap@lage.lysator.liu.se (H}kan Ander ss > on) writes: >> ... >>But have anyone thought of the similarity between cyberspace and >>the GUI's of modern computers? > > Yes. (At least, *I* have -- and I assume others as well). > >>In a sense, cyberspace is already invented, but today it's only two- >>dimensional, it sits on the screen in front of you. Yes, that's true if by Cyberspace you mean a user interface to an inrformation space which presents a abstract view of that space. But there's another element which we've talked about under the name of Virtual Reality: making that view of the information active. Right now, user interfaces are too passive (with respect to the information, not the user); the user can manipulate the objects represented in the interface, but the objects have trouble "manipulating back". In other words, there isn't enough sensory feedback from the object to give the user a sense of the change in the object other than the same cognitive feedback we find in textual interfaces. One purpose of datagloves, for instance, is to engage the sense of touch in the interface, to add to that sensory feedback (in addition to the added manipulation it allows the user). Even if there is no force or texture feedback in the gloves, the proprioceptive feedback of joint position and muscle pair contraction/distension adds a physical element to the interface which increases the sense of the "realness" of the objects being manipulated. > > Right. The extension to three dimensions is a critical element, as is the > notion of "sharing" that world with other people. I agree, but I think that the more "physical" the interface gets, the better, even if the graphics are less than ideal. Fancy 3D shaded graphics are nice, but a flat-colored object I can pick up is more real to me than a specular object my hand passes through. -- --------------------------------------------------------------------------- NOTE: USE THIS ADDRESS TO REPLY, REPLY-TO IN HEADER MAY BE BROKEN! Bruce Cohen, Computer Research Lab email: brucec@tekcrl.labs.tek.com Tektronix Laboratories, Tektronix, Inc. phone: (503)627-5241 M/S 50-662, P.O. Box 500, Beaverton, OR 97077
rnm@uunet.UU.NET (Robert Marsanyi) (09/17/90)
re: bandwidth of information and attention: that's a good point. Psycho- acoustics points to thresholds of rates of rates of change in phenomena as attention-grabbers, so using this measure to decide what to send around might help. --rbt (robert@f24.n125.z1.fidonet.ORG)
cyberoid@milton.u.washington.edu (Robert Jacobson) (09/17/90)
As Fred Brooks suggested in his paper at SIGGRAPH, "Project GROPE-- Haptic Displays for Scientific Visualization," there is no substitute for multisensory apprehension of data. Putting a complex image on a CRT merely displaces the process of making sense of an abstraction from textual translation to visual translation. Without a sense of spatiality or encompassing three-dimensionality, certain symbioses of the senses -- sound complementing sight, and body motion comple- menting both of those -- simply can't kick in. The oscillating glasses/CRT image concept is very creative and it produces a fine illusion, but you're still peeking through the porthole. You can't hear the waves lapping at the hull or walk the decks of experience. Bob Jacobson HIT Lab Seattle
xanthian@zorch.SF-Bay.ORG (Kent Paul Dolan) (09/21/90)
cyberoid@milton.u.washington.edu (Robert Jacobson) writes: > > > >As Fred Brooks suggested in his paper at SIGGRAPH, "Project GROPE-- >Haptic Displays for Scientific Visualization," there is no substitute >for multisensory apprehension of data. Putting a complex image on a >CRT merely displaces the process of making sense of an abstraction >from textual translation to visual translation. Without a sense of >spatiality or encompassing three-dimensionality, certain symbioses >of the senses -- sound complementing sight, and body motion comple- >menting both of those -- simply can't kick in. > >The oscillating glasses/CRT image concept is very creative and it >produces a fine illusion, but you're still peeking through the >porthole. You can't hear the waves lapping at the hull or walk >the decks of experience. > >Bob Jacobson >HIT Lab >Seattle Perhaps not, but the illusion of motion in a common arcade game is quite strong, even though the body is essentially still. Just like the hand eye coordination between a mouse and a screen cursor, I'd guess your body could quickly learn to compensate for missing cues from the motion sensors if the visual cues are made strong enough (3D, etc.). Thus, the virtual reality platform need not give you Mary Martin's flight support system to give the illusion of flight. Compare the truely limited real motion of the Disney starbus ride with the tremendous roller coaster sensation the intense visuals and minor motion cues provide. To generalize, don't overdesign the early systems to provide every possible sensory input. Start by trusting the body/memory/imagination/wishful thinking to provide the parts that are hard to simulate if you only do the easy ones well. Similarly, moving a joystick rather than ones head to look left or right may seem klutzy at first, and seem to demand the CRT be replaced by 3D head motion sensors and projection goggles in the initial design, but try the cheap, easy photo-optic goggles and interlaced left and right 3D views CRT screen solution, push it to the max, and see how the user adapts before deciding you have to invent the direct neural interface just because it's "the techy thing to do". ;-) Kent, the man from xanth. <xanthian@Zorch.SF-Bay.ORG> <xanthian@well.sf.ca.us>
cyberoid@milton.u.washington.edu (Robert Jacobson) (09/24/90)
In our discussion of how complex virtual worlds should be,
Kent makes the point:
>> Thus, the virtual reality platform need not give
you Mary Martin's flight support system
to give the illusion of flight. Compare the truely limited real motion
of the Disney starbus ride with the tremendous roller coaster sensation
the intense visuals and minor motion cues provide.
Don't underestimate the "limited" movement of the flight deck
in StarTours. I think it provides an incredible charge to the
visuals, as well as the other way around.hlab@milton.u.washington.edu (Human Int. Technology Lab) (09/26/90)
In article <7988@milton.u.washington.edu> xanthian@zorch.SF-Bay.ORG (Kent Paul D olan) writes: > Compare the truely limited real motion > of the Disney starbus ride with the tremendous roller coaster sensation > the intense visuals and minor motion cues provide. "Minor" motion cues? Those aren't so minor, Kent, because they're *acceleration" cues. That's exactly what would be missing if you don't have the Mary Martin flight harness. Think back on the Disney ride, and you`ll remember times when you woudl have sworn you were accelerating forward at nearly half a gee. It wasn't the visuals that did that, it was the fact that the hydraulic pillars holding up the cabin had gone into a backward-leaning position, so that you were lying on you rback, interpreting the resultant weight as forward acceleration. > To generalize, don't overdesign the early systems to provide every possible > sensory input. Start by trusting the body/memory/imagination/wishful > thinking to provide the parts that are hard to simulate if you only do the > easy ones well. This is good advice in any situation, but my own opinion is that there's a certain minimal functionality you need. That probably does not include force-feedback, but, again in my opinion, does include heads-up display. > > Similarly, moving a joystick rather than ones head to look left or right > may seem klutzy at first, and seem to demand the CRT be replaced by 3D > head motion sensors and projection goggles in the initial design, but try > the cheap, easy photo-optic goggles and interlaced left and right 3D views > CRT screen solution, push it to the max, and see how the user adapts before > deciding you have to invent the direct neural interface just because it's > "the techy thing to do". ;-) Well, we've all been talking as if this was a brand-new field, and no real research had been done in it. Not true: even the NASA-Ames teleoperator work was not the first. Most of the initial work in marrying human motor-sensory sytems to computer interfaces was done by the Navy adn the Air Force, in learning how to design control systems for submarines and fighter planes. Fighters are probably the paradigmatic application at this point in time: time-critical, mission-critical (!) tasks in which information overload is a fact of life, and the system MUST allow the person controlling the system to use instinct and trained reflex rather than symbol-cognitive reasoning. Heads-up displays are necessary in new fighter designs in part because some of the job is "out there" beyond the cockpit, and can't be coaxed into coming inside and tamed into the control system, but also because it gives the display system more of the pilot's visual field to display on, without permanently blocking the rest of the world. Virtual reality has a problem even beyond that of airplane control, one first noticed in teleoperator work: the operator needs to have a sense of identity with the "operand". Having a fixed viewpoint makes that identification more difficult, and also increases the stress on the operator as she tries unconsciously to "peer around" the viewpoint. -- --------------------------------------------------------------------------- NOTE: USE THIS ADDRESS TO REPLY, REPLY-TO IN HEADER MAY BE BROKEN! Bruce Cohen, Computer Research Lab email: brucec@tekcrl.labs.tek.com Tektronix Laboratories, Tektronix, Inc. phone: (503)627-5241 M/S 50-662, P.O. Box 500, Beaverton, OR 97077