bruceb@fai.UUCP (Bruce Bailey) (05/22/91)
Hi. I've been developing a c++ GUI toolkit that supports both mono and stereographic windows on a high-end SGI platform. I'm interested in sharing ideas, experiences, technical references, etc from interested people out there in net-land. In my experience, commercial GUI toolkits (regardless of the language/software design paradigm) fall down in a stereo environment. The reason is that (assuming a StereoGraphics Corp. hardware scenario) the user-interface entities need to appear simultaneously in both the bottom and top halves (left and right eye views) of the screen. To make matters worse, these 2 corresponding UI entities must appear synchronized in each view (imagine a pulldown menu in 2 different views, where buttons are simultaneously highlighted and unhighlighted as the cursor moves up and down). Another similar nightmare occurs with the cursor, since my (and probably most other) graphics hardware cannot support a dual-view cursor setup. Basically, I've been handling the above problems by creating classes for windows, user-interface entities, display list, and so on. When a mono window becomes stereo it takes over the entire screen (as per the hardware requirements). The only thing that needs to worry about stereo/mono display is the window object itself, which resolves stereo/mono display issues. The cursor, menus, display list, and so on are all passes to the window for display. I've addressed the cursor problem by implementing a software cursor, which is just another object derived from the UI class. Because I didn't want to write a full blown window system supporting both stereo and mono modes, I decided to go with sort of a poor-man's windowing environment instead. My windows support multiple viewports, where each viewport has its own display list, transform stack, and so on. Then, when the window becomes stereo (full screen), whichever viewport the cursor is in becomes "active", i.e., user commands are applied to the stuff in that viewport. Viewports cannot overlap. Currently, all my user interface objects are 2D and displayed at the plane of the screen. I'm toying with the possibility of going to 3D UI objects, which raises fascinating possibilities. For example, a 3D cursor controlled by a 3 axis input device, where the cursor is displayed in stereo. If the user pops up a menu, the menu could pop up out in front of the screen! Or, if the cursor is deep in the space of the display device and the user invoked a pulldown menu, the menu would come up at the same Z-depth as the cursor. This raises many design issues. Should UI objects in 3-space be z-buffered against stuff in the display list (conflicting cues may exist otherwise, e.g., stereo cues versus z-depth occlusion)? If z-buffering is used, the user would then be forced to move the UI object to an unobstructed area of the viewport/ screen. Or, suppose a decision is made to pop all UI menus out in front of the screen to avoid the occlusion/stereo cue conflict. With a pulldown menu the cursor would have to be popped out to the same z depth as the menu. Would this drive users crazy? Please respond with any ideas/experiences/questions you have. I'll summarize comments for possible future posting (if there is sufficient interest). Thanks in advance! Bruce Bailey (Dr. OW) Computational Research Dept. Fujitsu America, Inc. bruceb@fai.com
rthomson@mesa.dsd.es.com (Rich Thomson) (05/24/91)
In article <3066@fai.UUCP> bruceb@fai.fai.com (Bruce Bailey) writes: > In my experience, commercial GUI toolkits (regardless of the > language/software design paradigm) fall down in a stereo environment. I think it all depends on what kind of stereo environment your vendor supplies. The stereo environment supplied on our ESV workstations allows application developers to use all existing UI technology available with X, including Motif and Athena widget sets without any modification to source code, either the user's or the toolkit's. > The reason is that (assuming a StereoGraphics Corp. hardware > scenario) the user-interface entities need to appear simultaneously > in both the bottom and top halves (left and right eye views) of > the screen. To make matters worse, these 2 corresponding UI entities > must appear synchronized in each view (imagine a pulldown menu in > 2 different views, where buttons are simultaneously highlighted > and unhighlighted as the cursor moves up and down). Another similar > nightmare occurs with the cursor, since my (and probably most other) > graphics hardware cannot support a dual-view cursor setup. That is why we provided the concept of multiple logical screens for a single physical screen. These screens appear to the user as display:0.0, display:0.1, display:0.2, etc. At server startup time, the user is able to specify the number of screens (as an N by M array), the default visual type for each screen (8-bit PseudoColor, 24-bit TrueColor, 24-bit DirectColor) and a possible stereo mode for the screen (we support two stereo modes: one of resolution 640x512 with 1:1 aspect ratio pixels, and another of 1280x512 with 1:2 aspect ratio pixels). We provide an X extension that allows "warping" between logical screens, as well as a client (csm - Cary's Screen Manager) that allows selection of the different screens interactively. Another client is more suitable for a command-line interface (or in a window manager menu). Warping between screens by moving the cursor off the edge can be toggled on and off as well. For stereo screens, the X server renders all X graphics in the left and right-eye display buffers. This duplicates all "2D" graphics in each eye, including window manager menus, etc. For 3D graphics (supplied on ESV via a high-performance implementation of the PEX extension), a GSE (generalized structure element) instructs the server which two different view representations to use for the left and right eyes. > Basically, I've been handling the above problems by creating classes > for windows, user-interface entities, display list, and so on. In other words you're being forced to re-invent the wheel? > When a mono window becomes stereo it takes over the entire screen > (as per the hardware requirements). Sounds unfriendly in an X environment. What do you do about multiple stereo clients running at the same time? Multiple stereo windows at the same time? With our logical screen approach, none of these are a problem and no client has to "take over the entire screen". Running multiple stereo 3D clients (a single client with multiple 3D stereo windows) is no problem. Furthermore, they all interact with the window manager in the way one would expect. > Because I didn't want to write a full blown window system supporting > both stereo and mono modes, I decided to go with sort of a poor-man's > windowing environment instead. Again, sounds like you're having to re-invent the world in order to use stereo the way one would expect. > Bruce Bailey (Dr. OW) > Computational Research Dept. > Fujitsu America, Inc. > bruceb@fai.com -- Rich -- ``Read my MIPS -- no new VAXes!!'' -- George Bush after sniffing freon Disclaimer: I speak for myself, except as noted. UUCP: ...!uunet!dsd.es.com!rthomson Rich Thomson ARPA: rthomson@dsd.es.com PEXt Programmer