andyrose@batcomputer.tn.cornell.edu (Andy Rose) (11/14/90)
Craig Upson asks "does the 'visual programming' paradigm work?"
The visual programming paradigm is a perhaps recent development. It
basically refers to the ability to prototype code by "dragging" or
manipulating icons which represent functional modules on a "palette"
or workspace. Each icon has inputs and outputs. In AVS they are
represented as colored bars on the top and bottom of a rectangle which
contains the name of the module. The colors represent the type of data
which is needed or produced. In apE the icon is a picture and the name
can be displayed by pressing the middle button while in the icon. I
like the text approach better because of the difficulty of designing
meaningful ideograms for hundreds of modules.
The modules are "hooked together" using mouse interaction. In AVS
the user placed the mouse on an output color bar and holds a button down.
All the legal connections are displayed and the user moves the mouse to
highlight the one he needs. Disconnects are handled similarly.
The hooks, or "pipes" are color coded in AVS to indicate data type.
One of the unexpected advantages to designing code this way, besides
the implicit modular motif (call it object oriented, call it what you will)
is the appearence of parallel constructs. For example, if I wish to
extract edges from a 2D image, reverse the red and green from the original,
and filter the original, then add all the results or whatever, by "drawing"
the "network" to do this, the (in this case easy to comprehend) parallelism
falls out. In fact, you can "see" how parallel your code is.
A more obvious advantage is that there is virtually no compilation time since
all the modules are already compiled. Creating, editing, and testing
networks is a more free-flowing process than conventional edit, compile, link,
run, debug, edit...
Since modules can be of any complexity (ray tracers or AND gates), overhead
for network management determines granularity of modules.
Because a typical visualization task involves data reading, data conversion,
selection of appropriate visualization technique, various types of
interaction, and output, the visual programming paradigm is effective simply
because of its flexibility.
Most importantly for any one who has trained an inexperienced user, visual
programming is EASY (relatively). No operating system (not really), no
compiling, no code [best, rarest case] The more effective platform, AVS
over apE and khoros, has the better visual interface. AVS uses color
to indicate data type and the number of inputs and outputs are clearly
indicated. Because of their willingness to attempt machine portability,
apE and khoros rely on Xservers which right now are not fast enough for
many 3D applications, but they will get faster.
The "visual programming" paradigm works.
-
He asks "what's lacking from all these systems?"
The greatest kick down in the global research environment will come when these
systems are truly remote and distributed. I would like to double click
an AVS icon and see a list of machines on my network where this module
will run. Or, better, the system will balance the loads on the machines to
give peak or fairest performance. The pipes would indicate transfer speed
visually with thickness or color. If I have a ps/2 with vga or something
like that I wish to design nets. I shouldn't have to have a bigbuck machine
to do simple box dragging. Beyond the distributed wish, I would like a user
"community" to have access to my net. An example may clarify.
Dr. Bob has cool science code to visualize mass transport by fluid flow and
wants Dr. Sue to see it. Now Bob can't describe his science in words so
he videotapes (ack) the screen and sends it to Sue via snail-mail. Sue
calls him up and suggests he change the incident angle of the wave, which
he does and send her another tape.
It would be ideal if Bob got his simulation running with some interactive way
to change the incident angle (a dial on the screen) and calls Sue. Sue
sparks her visual server so she can watch what Bob sees. Sue is in San Diego,
BTW, and Bob is at Cornell. Compression schemes and high-speed nets
will help this. Now Sue can change the widget or Bob can change the widget.
If I were teaching a class on the subject I could lock out student access
to the widgets.
What this all begins to sound like is (oh god i hope this isn't a can of worms)
v__t__l r__l_ty. Bob's graphics (really three or four hundred spheres moving
in time) are represented in some database which Sue's server and Bob's server
can both see. Sue has signed on to Bob's simulation so she gets whatever
widgets are available to her.
Since the graphics are drawn locally (if Sue's machine is SGI fast), she
can select her own visual technique (radius, color of spheres, 3D mesh,
whatever) and the point of view. Bob may know that Sue is watching
because he sees an icon which represents Sue's point of view (an eyeball
with wings coming out of it suspended in a tire). Perhaps Sue's face
is hanging off the tire (a little bitmap).
Given processing speeds, network speeds, and enough disk, this could all be
done.
A lot of the solution to this has nothing to do with graphics. Alas.
--
Andrew Newkirk Rose '91 Department of Visualization CNSF/Theory Center
632 E & T Building, Hoy Road Ithaca, NY 14583
607 254 8686 andy@cornellf.tn.cornell.eduwave@media-lab.MEDIA.MIT.EDU (Michael B. Johnson) (11/14/90)
In article <1990Nov13.211634.28152@batcomputer.tn.cornell.edu> andyrose@batcomputer.tn.cornell.edu (Andy Rose) writes: >>Craig Upson asks "does the 'visual programming' paradigm work?" >> >>The visual programming paradigm is a perhaps recent development. It ... a nice description of the way AVS handles this stuff... >>The "visual programming" paradigm works. Well, yes and no. First of all, the coarseness of the networks being built hides an obvious lack - functional abstraction. Visual programming is a wonderful prototyping environment, but when it comes time to use it to build a large system, you really want the ability to "black-box" networks. What begins to happen is that you want to functionally abstract certain subnetworks into modules in a larger network. Also, the graphical programming paradigm is similar to an interperter for a conventional computer language. It's easy to change things, try things out, and you pay a certain performance penalty. It would be nice if there were "AVS/apE/etc compilers", that is, something which could take a network description and link the code appropriately such that you have a single efficient program to run. Note that I'm not saying this would necessarily be a single executable on a single machine - rather you could have network hooks between programs on seperate machines, but for those which are on the same machine, you could take advantage of shared memory and other such niceities to speed up computation and lessen the gratuitous data movement. >> >>He asks "what's lacking from all these systems?" >> >The greatest kick down in the global research environment will come when these >>systems are truly remote and distributed. I would like to double click >>an AVS icon and see a list of machines on my network where this module >>will run. Or, better, the system will balance the loads on the machines to >>give peak or fairest performance. The pipes would indicate transfer speed >>visually with thickness or color. If I have a ps/2 with vga or something >>like that I wish to design nets. I shouldn't have to have a bigbuck machine >>to do simple box dragging. Beyond the distributed wish, I would like a user >>"community" to have access to my net. >> Absolutely. My personal vision of this that a researcher will describe their problem in a virtual laboratory, using any of many available "problem description tools" (data flow diagrams, Mathematica, etc.). The computation of that problem would be handled by, for want of a better term, autonomous intelligent agents. I could go on, but some other time... >>A lot of the solution to this has nothing to do with graphics. Alas. As a fellow PhD student in my group said to me the other day, "Ya know, we must really love graphics, since we spend so much of our time working on other things." >>-- >>Andrew Newkirk Rose '91 Department of Visualization CNSF/Theory Center -- --> Michael B. Johnson --> MIT Media Lab -- Computer Graphics & Animation Group --> (617) 253-0663 -- wave@media-lab.media.mit.edu
scp@acl.lanl.gov (Stephen C. Pope) (11/14/90)
on 13 Nov 90 21:16:34 GMT, andyrose@batcomputer.tn.cornell.edu (Andy Rose) said: [ ... much deleted ] Andy> The greatest kick down in the global research environment will come when these Andy> systems are truly remote and distributed. [ ... ] Andy> It would be ideal if Bob got his simulation running with some interactive way Andy> to change the incident angle (a dial on the screen) and calls Sue. Sue Andy> sparks her visual server so she can watch what Bob sees. Sue is in San Diego, Andy> BTW, and Bob is at Cornell. Compression schemes and high-speed nets Andy> will help this. Now Sue can change the widget or Bob can change the widget. Andy> If I were teaching a class on the subject I could lock out student access Andy> to the widgets. [ ... ] Andy> Given processing speeds, network speeds, and enough disk, this could all be Andy> done. Folks at NCSA are already working on/doing this sort of stuff in conjuction with the gigabit testbeds. I think the lingo is something like ``collaborative visualization''. stephen pope advanced computing lab, lanl scp@acl.lanl.gov
pfeiffer@nmsu.edu (Joe Pfeiffer) (11/15/90)
In article <1990Nov13.211634.28152@batcomputer.tn.cornell.edu> andyrose@batcomputer.tn.cornell.edu (Andy Rose) writes:
Craig Upson asks "does the 'visual programming' paradigm work?"
The visual programming paradigm is a perhaps recent development. It
basically refers to the ability to prototype code by "dragging" or
manipulating icons which represent functional modules on a "palette"
or workspace. Each icon has inputs and outputs. In AVS they are
represented as colored bars on the top and bottom of a rectangle which
contains the name of the module. The colors represent the type of data
which is needed or produced. In apE the icon is a picture and the name
can be displayed by pressing the middle button while in the icon. I
like the text approach better because of the difficulty of designing
meaningful ideograms for hundreds of modules.
The modules are "hooked together" using mouse interaction. In AVS
the user placed the mouse on an output color bar and holds a button down.
All the legal connections are displayed and the user moves the mouse to
highlight the one he needs. Disconnects are handled similarly.
The hooks, or "pipes" are color coded in AVS to indicate data type.
<excellent comments on visual programming deleted>
While everything Andy says is true, it is important to note that
what he describes is not ``all'' of visual programming. It is,
rather, a member of a subclass I like to refer to as data flow visual
programming; other members of the subclass include Petri net
programming and state machine programming. One writes a program by
developing a data structure of some sort on the screen, and having
tokens flow between the elements of the structure.
More generally, visual programming refers to an environment where the
spatial relationship of objects on the screen describes an algorithm;
I'm working on an environment using graph grammars for data structure
manipulation, for example. It is possible for the perverse to make an
argument that textual languages fit this definition; this is not true
because the meaning of the position of symbols in a textual language
is one-dimensional. Languages where indentation level is significant
(such as Occam) start to fit this definition, but they are
abominations.
We now return you to your regularly scheduled discussion of
visualization. I've crossposted this to comp.lang.visual, and
directed followups there. I will also retire to my closet for my
asbestos suit, since I expect I'll be hearing from both Occam
supporters soon.
-Joe.