TKD@A.ISI.EDU (Thomas Donaldson) (06/29/89)
[Copied off the cryonics list, since Ralph's article also appeared here. --JoSH] This is a commentary on Ralph Merkle's "Computational Power" article. ABUSES OF POWER: COMPUTATIONAL POWER AND OUR BRAIN I read Ralph Merkle's paper with some interest. If I examined his figures closely I might find reasons to doubt his estimate by as much as a factor of 10, but that is trivia. However his paper does (implicitly) raise a central issue much more important: what is the SIGNIFICANCE of his computational power estimate, anyway? Ralph or anyone can spend much effort making a numerically and biologically valid estimate of almost any number we care to name. We can compute the topological index of the human brain, its area when spread over a flat surface, its reflectivity to light. Without explicit discussion of the significance of such measures the effort is incomplete. For several years now I've been involved directly in the parallel computer industry. When I originally got my doctorate, Computer Science as a discipline did not exist, nor did most of the theory of computation and programming developed since then. After spending a long time on theoretical mathematics (even writing one book), I worked for over 3 years with one company, SAXPY, which fell apart (I believe because of too much salesmanship combined with too little substance, until too late). The SAXPY 1-M was what is known as a SIMD machine. I'm now working with another company developing software for a MIMD local memory machine, working much like the hypercube but probably far less expensive. It's a very different machine, different problems, different solutions. Even as someone developing algorithms and writing programs in the back room, I have not failed to notice what goes on in front. In front we have Marketing, consisting of gentlemen in fine suits. Marketing must compare "our" computer with "theirs". What measure does Marketing seize upon for comparison? "Computational Power", of course! What has come most to my mind when I observe this travelling circus is that of how crude, even stupid, a measure Computational Power actually is. For a long time, Marketing for the SAXPY 1-M trumpeted its capability of doing 1000 Megaflops far less expensively than Crays. What Marketing never explained was that it could only achieve that figure on a narrow range of problems. On others it might, on a good day, equal an old 8088 IBM PC. Its merits lay in the specific problems for which it was designed. If we deal solely with a von Neumann sequential machine Computational Power makes a lot of sense. The machine must proceed on a straight road. Its velocity on that road accurately measures its effectiveness. For parallel machines, including our brains and the IBM TF-1, there is no straight road or even any single road. One machine with officially vast CP looks woefully pathetic on some problems which another machine, with much smaller CP, races through. The relation is reversed if we change the problems. We have every reason to believe that the same will happen when we try to make some (arbitrary, officially with godlike CP) machine imitate a human brain. The issue becomes even deeper than this. Because most of our machines now are sequential, we can make a clear distinction between "software" and "hardware". The hardware is the machine which runs along the route set out by the software. For parallel machines, particularly those, like hypercubes, consisting of many processors with local memory communicating with one another, that elementary Computer Science distinction breaks down. Choosing the right interconnections between processors is at least as important as writing software modules which run on each one. We aim for a combined system achieving our goals. Imitating a human brain with such a computer is indeed VERY simple: except for the design of BOTH HARDWARE AND SOFTWARE, we can consider the problem solved. How many, then, will go away happy with a figure for Computational Power? I've personally found involvement with parallelism quite fascinating. It's also my personal belief that a LOT of the Computer Science theory developed for sequential machines simply does not apply to parallel machines. We need to work out new ways to structure software; old ideas aren't enough. We must develop a lot anew and throw Yourdon out the window. There's pioneering and lots of fascination. As for imitating the human brain, I don't think that SPECIFIC task would even be wise. Of course Ralph intends broader and more powerful abilities than that. If we keep our noses to the grindstone someday we'll look up and see we've achieved those aims. Computational Power, though, by then will be a forgotten concept except among historians of ideas. -------
merkle.pa@Xerox.COM (06/29/89)
[This is Ralph's reply to Thomas' response, which I also snatched off the cryonics list. --JoSH] Thomas Donaldson <TKD@A.ISI.EDU> said: >I read Ralph Merkle's paper with some interest. If I examined his figures >closely I might find reasons to doubt his estimate by as much as a factor of >10, but that is trivia. However his paper does (implicitly) raise a central >issue much more important: what is the SIGNIFICANCE of his computational power >estimate, anyway? There are several implications of such a number. The most important implication is simply that we will have hardware in the not-too-distant future capable of out performing the human brain. By a large margin. By margins so large, that the prospect is daunting. Thomas objects that we will also need the software. This is true. He objects that we will need to know how to connect a large number of gates into a device effectively able to perform the computations the brain can do (the computer "architecture"). This is also true. Both these problems will be solved. At a minimum, when we have a technology capable of imaging the human brain with molecular precision, we can determine the neuronal structure and function in full detail. We have already determined what the retina does with a fairly high accuracy. With further effort, we should at least be able to determine how the isolated parts of the brain work. Even if we are unable to see the "grand pattern," if we can analyze local structure we can then duplicate its local function. Then, we can connect these pieces together in the "grand pattern" even without understanding exactly how it works. The "grand pattern" might continue to be mysterious, but we would have produced human abilities in a computer. It seems quite likely that something more interesting will happen. We will probably solve the problems of AI. We can do so directly (without "cheating" and looking at the existing biological solutions) or we can do so indirectly. The second approach is more likely. In this approach, we combine everything we can guess or learn about the function of the biological system with everything we can guess or learn about machine intelligence by simply trying to write intelligent programs. As we learn more on either front, it provides a better and more coherent picture of the whole. Eventually, we will learn a significant amount about both, and combine our understanding of both into a very powerful thinking system. I am not impressed with the "Frankenstein's Monster" fear that such a system will turn against its masters. I am concerned that such a system will do more or less what it was designed and built to do. It is this second concern that seems more significant. The existence of the human brain sets lower bounds on what can be accomplished by a system of given complexity. We will be able to create systems of much greater complexity, therefore they will be able to do at least the following: 1.) Solve the same problems faster. The basic time delay in biological systems is one millisecond, while the basic time delay in computers of the future will be less than (and probably quite a bit less than) one nanosecond. Thus, we will be able to build devices that solve problems that human beings solve today, but they will be able to do so over one million times faster. 2.) Solve many instances of the same problem, independently and in parallel. Again, a factor of a million or more seems likely. Thomas argues that a specific computer, designed for a specific problem, will solve that specific problem faster and better than a different computer of similar complexity, but which was designed to solve a different problem. Hence, "benchmarks" are irrelevant. A computer that runs a particular benchmark very quickly as compared with another "slower" computer, might run a different benchmark very slowly. This is true, but largely beside the point. He also argues that much of what is known about conventional serial computation will not prove useful in a parallel computation. This is almost certainly true for some range of computational problems. However, if a task is already performed by the human brain, then we know a lower bound on how well that task can be performed by a device specifically designed to solve that task. A device with one million times the raw computational capacity of the human brain, specifically designed to solve a problem already solved by the human brain, will do so one million times more quickly, or solve a million instances of the problem while the human could solve only one. A number of specific possibilities will no doubt come to mind. Computational power a forgotten concept? No. If I have more gates, operating faster, I can compute more things more quickly. How to arrange the gates to best advantage to solve a particular problem will continue to be a problem of great interest -- but if I have a million times as many gates, operating a million times faster, then the device I build will compute at least a million times faster and can do so a million times in parallel while your device can finish but one problem. (Assuming I can peek at your design and steal the good ideas you use. Which, in the case of the human brain, we will be able to do at some point). The significance? We will build such devices, and use them to accomplish ends of our devising. We will build them, and set them some tasks. What tasks? Perhaps we need to think about this. What ends? Perhaps we should be careful. And, perhaps most vexing, WHO will set these tasks? It might not be us. It might not even be someone we like very much. What then? Perhaps it would help if we thought about these things before they happened. Of course, to think about such things, we must first be aware that they are possible. Hence my paper.