[comp.parallel] terminology

eugene@wilbur.nas.nasa.gov (Eugene N. Miya) (06/07/90)

In article <9223@hubcap.clemson.edu> art@cs.bu.edu (Al Thompson) writes:
>It's an arbitrary thing.
>They are arbitrary, but they make a certain amount of sense.
>they are as good as anything else.
>other serious scientist) would not consider a machine that synched at 19
>instructions to be fine grained while one that synched at 21 to be medium.
>It doesn't make any sense except to the anally pedantic.  If you are going
>to have terms like "grain" floating around it's a good idea to have AGREED
>(not natural, whatever that means) definitions.  Agreed, simply so that
>when we encounter the terms in isolation from other scientists we will
>"know" what they mean.
>
>One of the problems in computing is this lack of well defined terms.  As a
>scientist trained in another discipline (biophysics) I am continually
>appalled at the lexical anarchy in the computing "sciences". << AGREED.
>In fact I
>was so startled by this that I undertook a study of just what it is that
>constitutes "knowledge" from the point of view of an established science.
>What comes through clearly is the newness of computing.	<< AGREED.
>KUHN.  He makes the important point that you know a
>true paradigm has been defined when the basic definitions appear in the
>"standard" textbooks.  This has the effect of schooling a generation of
>scientists who think that the words they use are somehow invariant.  This
>is not true, but it does start them off that way.  Only later when they
>are faced with paradigm crisis and shift do they truly confront this
>issue.  I suggest that that is exactly where "computer science", "computer
>engineering" and "computational science (the new kid on the block)" are
>today.  If you don't believe me, look at the textbooks, particularly look
>for agreement from author to author.  The more agreement you see the
>closer we are.  Now, as an exercise, look at the books of twenty years
>ago and repeat the comparison.

I just posted a note about this with regard to the recent architecture
symp. (in comp.arch).  More respected architects are also encountering
this communications problem.  They too are having he problem with PAPERs
and TEXTS.  The purpose of my note was that I didn't
care for the emphasis on synchronizing cycles, too control flow oriented.
And if you have never studied dataflow where the synchronizations
can occur many cycles away.....  I hope you get the picture.

The problem isn't simply terminology (I think enough readers of
comp.parallel have seen me post on the problem of terminology).
Comp.arch is having another "super-linear speed-up" discussion,
another use of bad terminology.  But consider that some concepts
are completely foreign to some people: imagine working on a word oriented
machine all your life then working on a byte oriented machine.
See my comp.arch posting if this example is too vague.

I do not think we should be too arbitrary.  We must be careful and we
will get burned on occasion.  Fortran was somewhat arbitrary, we didn't
know about programming languages back then.  We know more now.  Backus
and others thought one could make a PL based on simple algebra (not enough).

I came from math (and nuclear engineering).  Our use of language has
become so precise it's stilting: an "ideal" probably means one thing
to most people, but mathematicians its meaning is "refined."  This is
offered as a comment to your comment on invariance.  It's not simply
invariant or arbitrary.

I empathize with your frustration.  Computer science isn't.  Part of
the problem is simultaneous over- and under- emphasis on math
and under emphasis on empirical work/construction.  This is changing.
There are also economic considerations and in the high performance
arena, there are "political" considerations.  There are companies
(some read the net, others don't) whose best economic interests
are served by maintaining confusion.  That's how economic works 8).
That's not science.  I only point to benchmarking as one example of this
problem (i.e. performance measurement).


Your moderator (Steve) pointed me to an excellent book: The Language of
Thought (which I'm picking away at).  Remember that "parallel computing"
isn't, either.  It's parallel, part of the time.  My officemate George
is always harping on me about "notation" with Whitehead as an example.
We do have to agree upon definitions so we can move on,
economic/political considerations be damned.  If we want science, if we
want faster machines, etc.

Keep policing the language.

--e. nobuo miya, NASA Ames Research Center, eugene@orville.nas.nasa.gov
  {uunet,mailrus,other gateways}!ames!eugene

art@cs.bu.edu (Al Thompson) (06/07/90)

In article <9233@hubcap.clemson.edu> eugene@wilbur.nas.nasa.gov (Eugene N. Miya) writes:
|In article <9223@hubcap.clemson.edu> art@cs.bu.edu (Al Thompson) writes:
|>It's an arbitrary thing.
|>They are arbitrary, but they make a certain amount of sense.
|>they are as good as anything else.
|>other serious scientist) would not consider a machine that synched at 19
|>instructions to be fine grained while one that synched at 21 to be medium.
|>It doesn't make any sense except to the anally pedantic.  If you are going
|>to have terms like "grain" floating around it's a good idea to have AGREED
|>(not natural, whatever that means) definitions.  Agreed, simply so that
|>when we encounter the terms in isolation from other scientists we will
|>"know" what they mean.
|>
|>One of the problems in computing is this lack of well defined terms.  As a
|>scientist trained in another discipline (biophysics) I am continually
|>appalled at the lexical anarchy in the computing "sciences". << AGREED.
|>In fact I
|>was so startled by this that I undertook a study of just what it is that
|>constitutes "knowledge" from the point of view of an established science.
|>What comes through clearly is the newness of computing.	<< AGREED.
|>KUHN.  He makes the important point that you know a
|>true paradigm has been defined when the basic definitions appear in the
|>"standard" textbooks.  This has the effect of schooling a generation of
|>scientists who think that the words they use are somehow invariant.  This
|>is not true, but it does start them off that way.  Only later when they
|>are faced with paradigm crisis and shift do they truly confront this
|>issue.  I suggest that that is exactly where "computer science", "computer
|>engineering" and "computational science (the new kid on the block)" are
|>today.  If you don't believe me, look at the textbooks, particularly look
|>for agreement from author to author.  The more agreement you see the
|>closer we are.  Now, as an exercise, look at the books of twenty years
|>ago and repeat the comparison.
|
|I just posted a note about this with regard to the recent architecture
|symp. (in comp.arch).  More respected architects are also encountering
|this communications problem.  They too are having he problem with PAPERs
|and TEXTS.  The purpose of my note was that I didn't
|care for the emphasis on synchronizing cycles, too control flow oriented.
|And if you have never studied dataflow where the synchronizations
|can occur many cycles away.....  I hope you get the picture.

I do, which is why I cited the Stone definition.  He doesn't refer to
synchronization at all.  He defines the grain as a ratio of a run-time
quantum to a communication cost.  Now I'll admit it doesn't take a huge
leap to imply synch from that, but he does avoid explicitly mentioning it.

|
|The problem isn't simply terminology (I think enough readers of
|comp.parallel have seen me post on the problem of terminology).
|Comp.arch is having another "super-linear speed-up" discussion,
|another use of bad terminology.  But consider that some concepts
|are completely foreign to some people: imagine working on a word oriented
|machine all your life then working on a byte oriented machine.
|See my comp.arch posting if this example is too vague.
|
|I do not think we should be too arbitrary.  We must be careful and we
|will get burned on occasion.  Fortran was somewhat arbitrary, we didn't
|know about programming languages back then.  We know more now.  Backus
|and others thought one could make a PL based on simple algebra (not enough).

I wasn't suggesting we should too arbitrary.  I was trying to point out
that all technical definitions are arbitrary.  But, that does not mean
they are random.  With any luck the arbitrary choices are made with some
attention to "naturalness".  But, we use technical terminology so that we
can include some things and exclude others.  Technical definitions by
design exclude some things from consideration even when they are close.  A
classical case is the definition of "species" in biology.  Oh my the heat
that generated (and to a certain extent still does).  Another case is the
debates between Bishop Berkeley and Newton over the idea of an
infinitesimal.  Berkeley picked infinitely fine nits.  Ultimately Newton
threw up his hands and got rid of the idea altogether.  The result is that
the fundamental definitions of derivatives and integrals are much tighter.

|
|I came from math (and nuclear engineering).  Our use of language has
|become so precise it's stilting: an "ideal" probably means one thing
|to most people, but mathematicians its meaning is "refined."  This is
|offered as a comment to your comment on invariance.  It's not simply
|invariant or arbitrary.

Sure, but is this a technical definition or one that has worked its way
into the jargon as an accepted meaning?  It matters because my point
concerns technical definitions (the jargon part is just as valid, but it's
much tougher to get a handle on it).

|
|I empathize with your frustration.

Ive gotten used to it.  I have to admit though that when I first started
doing computing I was an advanced grad student (two years to the Ph.D.)
I was blown away.

  Computer science isn't.  Part of
|the problem is simultaneous over- and under- emphasis on math
|and under emphasis on empirical work/construction.  This is changing.
|There are also economic considerations and in the high performance
|arena, there are "political" considerations.  There are companies
|(some read the net, others don't) whose best economic interests
|are served by maintaining confusion.

This is my product: it is a high friction cellulose aggregation apparatus.

This is my competitor's product:  it is a nail.

  That's how economic works 8).
|That's not science.  I only point to benchmarking as one example of this
|problem (i.e. performance measurement).
|
|
|Your moderator (Steve) pointed me to an excellent book: The Language of
|Thought (which I'm picking away at).  Remember that "parallel computing"
|isn't, either.  It's parallel, part of the time.

Parallel is a word that should should be banished except as it applies to
parking.  It is about as misleading a term as you can find.  It is so
general as to be nearly useless.  Parallel is to computing as plant and
animal is to biology.  A dung beetle is an animal, so is an elephant.  A
two processor Encore is a parallel computer, so is a 64k processor
Connection Machine.  Are there any other similarities?

  My officemate George
|is always harping on me about "notation" with Whitehead as an example.

Whorf comes immediately to mind here.  The Sapir(credit where credit is
due)-Whorf hypothesis has difficulties when you are discussing natural
language, but when applied to programming languages it can have real
utility.  Look at the different kinds of thinking you must employ when you
write a program in Lisp as compared to when you are writing in Pascal.
This despite the fact that you can get exactly the same results.

|We do have to agree upon definitions so we can move on,
|economic/political considerations be damned.  If we want science, if we
|want faster machines, etc.

Actually this will happen.  In truth it's really starting to happen.
You're seeing more and more consistency.  It's just seems to be a little
slow in coming.  Other disciplines went through the same thing.  the
difference is perception.  Those who were schooled in those disciplines
were brought up to expect a certain semantic consistency.  Suddenly
something new shows up and people need a jargon.

Your observation about the economic issue is important.  Computing is
probably the only discipline that has had this kind of economic pressure
from day one.  Other disciplines had time to put the house in order before
they became economically important.  Not so for computing.  So, it is
small wonder that things are confused.

|
|Keep policing the language.

I've nearly got enough to make an arrest.

bbc@rice.edu (Benjamin Chase) (06/08/90)

In article <9238@hubcap.clemson.edu> art@cs.bu.edu (Al Thompson) writes:
>I do, which is why I cited the Stone definition.  He doesn't refer to
>synchronization at all.  He defines the grain as a ratio of a run-time
>quantum to a communication cost.  Now I'll admit it doesn't take a huge
>leap to imply synch from that, but he does avoid explicitly mentioning it.

The problem I have with this definition is that "grain size" then
becomes architecture dependent.  If communication (eg.
synchronization) is poorly implemented (or cannot be implemented
efficiently on a particular architecture) and hence has a high cost,
then a grain in the "fine grain" world contains more instructions on
this system.

If we're voting on the meaning for this particular collection of
terms, I would rather that the size of the grain be keyed on something
like the number of instructions within a grain.  I feel that this
definition will prove more timeless than the other, as hardware and
software evolve.  In years to come, we won't find the need for
ultra-hyper-micro-fine grain, because an obvious lower limit for
instructions/syncronization is 1.  Also, with this choice of
definition, we find that some architectures (eg. loosely-coupled) are
inappropriate for fine-grain concurrency.  With the other definition,
we instead find that the fine grains on some architectures are in fact
quite coarse...
--
	Ben Chase <bbc@rice.edu>, Rice University, Houston, Texas