yba@athena.mit.edu (Mark H Levine) (01/16/88)
This is the second of two postings describing work at MIT in
interactive video disc related areas. It is a copy of the
article recently published in the first issue of the X User's
Group Newsletter. I hope the slant toward the X User audience
does not make it less useful.
Visual Courseware and the X Window System
In the excitement surrounding X today, it is easy to
forget that four years ago its development was partly
motivated by Project Athena's need to support teaching.
While most of the applications on the Athena system use only
text and graphics, ten development projects also use video.
This article presents a brief introduction to Athena's
effort to integrate X with video.
Video at the workstation allows a student to actually
see and hear native speakers on the streets of Paris while
studying French; to watch motion video of bearings operating
under conditions the student selects; to browse through an
interactive visual database of the human brain; and to
interact with the material through the many possibilities of
the computer, rather than just to view a video tape pas-
sively -- the student controls the language tape through a
natural (foreign) language recognizer. Video as a network
resource allows the sharing of still and moving picture
libraries, computer enhanced images, closed circuit televi-
sion and video recordings of all types.
At Project Athena, the Visual Courseware Group (VCG)
supports the integration of video material and optical media
into our workstation environment. Specifically, our research
and development efforts target the extension of the X Window
System to handle both motion and still video as the contents
of a window and network retrieval of pictures and other
visual materials. In general, VCG develops tools for the
preparation and use of video and image data as part of an
integrated display system that also includes both computer
text and graphics. To date we have successfully integrated
video with X Version 10 Release 4 (X10), and we are now
adding this capability to X Version 11 (X11).
VCG uses a "visual workstation" specially configured to
deliver these materials; this station includes a 1280x1024x8
RGB color display, three-button mouse, twelve-inch optical
disk players, stereo audio, and access to both the MIT cable
TV system and the Athena network. The visual workstation
may be based on either the DEC Microvax-II or IBM RT PC, and
depends heavily on the Model 1280 videographics processor
from Parallax Graphics, Inc.
The user of a visual workstation can see both moving
and still video as the contents of a window. Video windows
can be moved, resized, and iconified; they behave for the
most part like any other X window. Still frames can be
scaled and distorted by the resize operation, and both
motion and still pictures can be clipped to a window region.
Graphics can be drawn over video images.
Under X10, the programmer's interface to video exten-
sions is primarily contained in a set of four routines which
exploit a "hole" in the Xlib's error checking to communicate
with the X server for the Parallax 1280; they provide the
basic operations:
XStartVideo(w,srcpix,sx,sy,prio)
Windoww;
intsrcpix, sx, sy, prio;
{
if (!nullbm)
nullbm = XStoreBitmap(1,1,&sx);
XPixFill(w,0,0,2048|sx,2048|sy,0,nullbm,1,1);
XPixFill(w,0,0,2048|prio,2048,srcpix,nullbm,0,1);
XSync(FALSE);
}
XStartVideo () will start flashing live video into
existing X window "w" -- it allows the user to clear
the background to the pattern in "srcpix" and to choose
the upper left corner of the full size video source
frame which becomes the upper left corner of the
displayed window with "sx" and "sy". "prio" is a
number indicating the relative priority of graphics
versus video.
Similarly, XStillVideo() puts up a still frame,
and XScaleVideo () will shrink or expand a still frame
as it is displayed. Pseudo-color values which will be
drawn over video regions may also be set with a call to
XSetVideoColor ().
For use with X11, we plan to implement a more complete
video extension library.
The actual implementation of X10 on the videographics
processor was a joint effort by Robert L. Goodwin of Paral-
lax and the author. The first version was finished early in
1987 and has been a base for VCG applications work at MIT
since then. An X11 port begun last summer is still under
development.
Application work currently in progress includes:
1. A Display Handler to control presentation of visual
material based on plain text instructions (author-
ing support).
2. Tools for making menus and other (X Toolkit)
objects at a higher level than that provided by X.
3. Glossary support: tools to look up words presented
on the screen in a glossary and to present the
explanations found (for example, this will be used
in the subtitles of language films and to explain
medical jargon).
4. General text browsing and note-taking tools which
will connect to the glossary tools.
5. Image processing software to display parts of large
images or assemble large images from several parts
or frames (this will support enlargements of com-
plex images and the assembly of composites from
several sources).
6. "Tagging" software to associate "objects" with
parts of a picture.
7. A video-editing suite for use in retrieving and
compiling materials (preparation of reports and
lectures based on library materials, for example,
or the production of a video disk).
8. A timeline editor: for synchronizing time-dependent
elements in a multi-media display. For example,
text subtitles for a motion video, perhaps with
alternate sound tracks.
9. An image delivery service. This would, for
instance, provide retrieval of video and data
stored on optical disks from a central location
over the network to the video workstation (using
the baseband network for control and the broadband
network for transmission).
With such tools, we intend to demonstrate the impor-
tance of video materials and optical media as a part of both
educational and workstation technologies, and to simplify
authoring of applications which use these technologies.
I would like to personally thank Ken Irvin of MITRE,
Frank Janes of NASA, Lester Ludwig of BellCoRe, and all at
Parallax for their encouragement and support of the X11
effort.
Further information about Athena may be obtained and
visits may be arranged through contacting Dr. Kenneth A.
Goldman, E40-358F MIT, Cambridge, MA, 02139 [e-mail:
democenter@athena.mit.edu; telephone: (617) 253-0194].
Further information about Parallax products can be obtained
from Marty Picco, Parallax Graphics Inc., 2500 Condensa
Street, Santa Clara, CA, 95051 [e-mail:
picco%parallax.uucp@sun.com; telephone: (408) 727-2220].
The author is solely responsible for the accuracy of
the information given; no statement should be construed as
necessarily representing the position of any of the organi-
zations mentioned.
(The author is I).