andrew@berlioz (Lord Snooty @ The Giant Poisoned Electric Head ) (08/04/89)
In article <24241@iuvax.cs.indiana.edu>, dave@cogsci.indiana.edu (David Chalmers) writes:
[A discussion on fads and the rapid growth of connectionism, and a prediction
of its demise through hype]
I think you should crosspost this to comp.ai.neural-nets, whose members seem
to exhibit the usual healthy cynicism of a comp.. group; not a pack of
zealots by any means!
I agree that there exists a danger from over-rapid over-exposure and
the concomitant media hype. This is a constant warning cry made at the
conferences, and by people who popularise the field. You have to bear in
mind that we're only human, and become naturally excited even as researchers
and informed observers when new results appear. It is not necessary to
*immediately* understand the nature of the underlying mechanism when
a new and successful application is created (in this sense, your analogy
to cold fusion is spot-on).
I think that what is required to save the field from the "hype seesaw"
is a healthy rate of generation of solid new theoretical results.
Two fairly recent results, for example, which could be seen to qualify:
1) A preprocessing paradigm using a simple one-layer net and an easily-
implementable learning algorithm, which extracts the eigenvalues of
the input autocorrelation - useful for image compression, etc.
In particular, information-theoretic approaches are producing new
results. [Sanger, Linsker, Foldiak]
2) A formal proof of an algorithm for a restricted class of nets, which
predicts detailed network dynamics given the training pattern set.
[Lemmon, Kumar]
There is a tremendous amount of high-quality work going on, bolstered by
the application of formal mathematical techniques.
It seems to me that this truly sets NN research apart from the much
more "hand-waving" stuff that I encountered when looking at conventional
AI, when expert systems were on the rise in the early- and mid-80s.
Here one found tree traversal stuff and Bayesian statistical variations,
definitons of "frames" and the like; the ad hoc component was significant.
(although fuzzy set theory has to some extent set some of this on a more
formal footing, I have to agree).
The analogy I have in mind equates NN research to the microstructure of
cognition, and as such is akin to "physics". When dealing with the atoms
of behaviour, it's possible to produce significant and fundamental results.
Symbolic AI smacks to me much more like "inorganic chemistry".
The consensus view seems to be that these two paradigms will eventually
cooperate in future artificial cognitive systems. Work is already
ongoing to combine expert systems with NN coprocessors. However, taking
the brain as an existence proof, it's clear that NN technology can
implement all levels of cognition, whereas it is unclear whether symbolic
methods are capable of this [see e.g. Steve Harnath: subsymbolic and
symbolic processing].
--
...........................................................................
Andrew Palfreyman There's a good time coming, be it ever so far away,
andrew@berlioz.nsc.com That's what I says to myself, says I,
time sucks jolly good luck, hooray!andrew@berlioz (Lord Snooty @ The Giant Poisoned Electric Head ) (08/04/89)
My apologies for the misspelling of the name of author Steve Harnad. I think that's right now. -- ........................................................................... Andrew Palfreyman There's a good time coming, be it ever so far away, andrew@berlioz.nsc.com That's what I says to myself, says I, time sucks jolly good luck, hooray!
holt@cs.AthabascaU.CA (Peter Holt) (08/07/89)
I would say that is a fairly zealous statement. Personally I have not decided which paradigm is better for what when yet, but lets remember that there may only be a superficial resemblance between the operations of the brain and current neural net technology! A lot more things are happening in the brain (especially chemically and at the intraneuron level) than are in neural nets. It may even be a coincidence that what some of the functionality of neural nets approximates some of the very basic perceptual-cognitive functions of the brain. Some of the other functionality of neural nets (extracting eigenvalues?) would not seem to match the way humans do same things at all.
andrew@berlioz (Lord Snooty @ The Giant Poisoned Electric Head ) (08/08/89)
In article <705@aurora.AthabascaU.CA>,holt@cs.AthabascaU.CA (Peter Holt) writes: > ..but lets remember that there may only be a superficial resemblance > between the operations of the brain and current neural net technology! Let's talk about "resemblance", then. "Resemblance" is a strong suit for nets in the connectionism vs. serial symbolic systems debate, and yet you use it for critique! When PROLOG executes a branch instruction in the ALU of the SPARC chip, where is the resemblance to the brain? -- ........................................................................... Andrew Palfreyman There's a good time coming, be it ever so far away, andrew@berlioz.nsc.com That's what I says to myself, says I, time sucks jolly good luck, hooray!
jps@cat.cmu.edu (James Salsman) (08/08/89)
In article <599@berlioz.nsc.com> andrew@berlioz (Lord Snooty @ The Giant Poisoned Electric Head ) writes: > When PROLOG executes a branch instruction in the ALU of the SPARC chip, > where is the resemblance to the brain? It depends on the rest of the SPARC system's state. If you have a formal description of a data structure and an algorithm, then you have a program. Using a technique called "programming" one may map these descriptions on to different kinds of computer systems. The Neural-Net of the brain is one kind of system, and a SPARC system is somthing else entirely. The only reason that they can't be executing the same program is that the I/O systems are very different. :James -- :James P. Salsman (jps@CAT.CMU.EDU)
davidvc@binky.sybase.com (David Van Couvering) (08/09/89)
In article <705@aurora.AthabascaU.CA> you write: >Personally I have not decided which paradigm is better for what when yet, >but lets remember that there may only be a superficial resemblance >between the operations of the brain and current neural net technology! >A lot more things are happening in the brain (especially chemically >and at the intraneuron level) than are in neural nets. >It may even be a coincidence that what some of the functionality >of neural nets approximates some of the very basic perceptual-cognitive >functions of the brain. Some of the other functionality of neural nets >(extracting eigenvalues?) would not seem to match the way >humans do same things at all. Hear hear! There is still so much to know/learn about the brain and human cognition. For instance, the actual mechanism for learning/memory. For instance, the actual mechanism for pattern recognition. For God's sake, we don't even know what 80% of the brain does! Not to put down the great progress of neural network technology, but to remember what a great realm of work there is still to do. David davidvc@sybase.com {pacbell, lll-tis, pyramid, sun}!sybase!davidvc
coggins@coggins.cs.unc.edu (Dr. James Coggins) (08/13/89)
In article <568@berlioz.nsc.com> andrew@berlioz (Lord Snooty @ The Giant Poisoned Electric Head ) writes: >I think you should crosspost this to comp.ai.neural-nets, whose members seem >to exhibit the usual healthy cynicism of a comp.. group; not a pack of >zealots by any means! Thank you, I'm sure. >I think that what is required to save the field from the "hype seesaw" >is a healthy rate of generation of solid new theoretical results. >There is a tremendous amount of high-quality work going on, bolstered by >the application of formal mathematical techniques. >It seems to me that this truly sets NN research apart from the much >more "hand-waving" stuff that I encountered when looking at conventional >AI, when expert systems were on the rise in the early- and mid-80s. >Here one found tree traversal stuff and Bayesian statistical variations, >definitons of "frames" and the like; the ad hoc component was significant. >(although fuzzy set theory has to some extent set some of this on a more >formal footing, I have to agree). > >Andrew Palfreyman There's a good time coming, be it ever so far away, >andrew@berlioz.nsc.com That's what I says to myself, says I, I'm afraid that the theoretical foundation you appreciate is actually inherited (or bastardized, depending on your point of view) from the statistical pattern recognition studies of ten to twenty years ago. Sure there is a theory base, but it's ready-made, much of it not arising inherently from NNs (but being REdiscovered there). "...only be sure please always to call it RESEARCH!" from Lobachevsky by Tom Lehrer I have been impressed with the confirmation provided by this newsgroup that the majority of researchers in this area really are disgusted at the publicity-mongering, money-grubbing approach of too many well-placed (and well-heeled) labs, researchers, writers, companies, seminar sellers, and the like. NNs might become a significant contribution making possible highly parallel implementations of many kinds of processes if the science fiction futurist brain-theory dabblers would shut up and let the real researchers develop the field in a careful, disciplined way, without having to run interference against massively inflated expectations of the work. A few months ago I posted to comp.ai.neural-nets the document reproduced below. I guess it was too hot for the newsgroup, but I did receive 13 e-mail replies: 8 firmly supportive, 4 asking for more pointers to statistical pattern recognition which I gladly supplied (But note: Is the scholarship in the NN field really so weak that NN researchers are unaware of twenty years of research in statistical pattern recognition? The evidence says yes!), and one sharply critical but easy to refute (a True Believer who went down in flames). I posted the document below in the spirit of my other "Outrageous Discussion Papers" that I have been circulating to carefully selected audiences to provoke thought and comment and encourage skepticism. I have one flaming the use of rule-based expert systems in medical applications, one arguing that edges are an inadequate foundation for vision, one arguing that automatic identification of organs in CT scans is an unworthy task of little practical value, one that is a manifesto for my approach to computer vision research, and the neural net one below. If you are interested, e-mail me, but I'm leaving now for a three-week vacation, so don't expect my usual rapid response. --------------------------------------------- My assessment of the neural net area is as follows: (consider these Six Theses nailed to the church door) 1. NNs are a parallel implementation technique that shows promise for making perceptual processes run in real time. 2. There is nothing in the NN work that is fundamentally new except as a fast implementation. Their ability to learn incrementally from a series of samples nice but not new. The way they learn and make decisions is decades old and first arose in communication theory, then was further developed in statistical pattern recognition. 3. The claims that NNs are fundamentally new are founded on ignorance of statistical pattern recognition or on simplistic views of the nature of statistical pattern recognition. I have heard supposedly competent people working in NNs claim that statistical pattern recognition is based on assumptions of Gaussian distributions which are not required in NNs, therefore NNs are fundamentally different. This is ridiculous. Statistical pattern recognition is not bound to Gaussians, and NNs do, most assuredly, incorporate distributional assumptions in their decision criteria. 4. A more cynical view that I do not fully embrace says that the main function of "Neural Networks" is as a label for money. It is a flag you wave to attract money dispensed by people who are interested in the engineering of real-time perceptual processing and who are ignorant of statistical pattern recognition and therefore the lack of substance of the neural net field. 5. Neural nets raise lots of engineering questions but little science. Much of the excitement they have raised is based on uncritical acceptance of "neat" demos and ignorance. As such, the area resembles a religion more than a science. 6. The "popularity" of neural net research is a consequence of the miserable mathematical backgrounds of computer science students (and some professors!). You don't need to know any math to be a hacker, but you have to know math and statistics to work in statistical pattern recognition. Thus, generations of computer science students are susceptible to hoodwinking by neat demos based on simple mathematical and statistical techniques that incorporate some engineering hacks that can be tweaked forever. They'll think they are accomplishing something by their endless tweaking because they don't know enough math and statistics to tell what's really going on. --------------------------------------------------------------------- Dr. James M. Coggins coggins@cs.unc.edu Computer Science Department A neuromorphic minimum distance classifier! UNC-Chapel Hill Big freaking hairy deal. Chapel Hill, NC 27599-3175 -Garfield the Cat and NASA Center of Excellence in Space Data and Information Science ---------------------------------------------------------------------
lee@uhccux.uhcc.hawaii.edu (Greg Lee) (08/14/89)
From article <9143@thorin.cs.unc.edu>, by coggins@coggins.cs.unc.edu (Dr. James Coggins):
>5. Neural nets raise lots of engineering questions but little science.
Judging from popular accounts, and as an outsider to the field, this is
the impression I get -- that NNs are an attempt to do technology without
science. I have seen what I take to be kindred approaches in my own
field, linguistics. The idea seems to be that one can escape the
necessity to achieve an understanding of human perception and leave that
to a machine (or algorithm, rather). Since scientific understanding
(new and old) is so difficult to come by, it's a very seductive idea.
But not a reasonable one.
Greg, lee@uhccux.uhcc.hawaii.edubph@buengc.BU.EDU (Blair P. Houghton) (08/14/89)
In article <4559@uhccux.uhcc.hawaii.edu> lee@uhccux.uhcc.hawaii.edu (Greg Lee) writes: >From article <9143@thorin.cs.unc.edu>, by coggins@coggins.cs.unc.edu (Dr. James Coggins): > >>5. Neural nets raise lots of engineering questions but little science. Eh? Science has been the forming of models and the fitting of them to observed phenomena. In the case of artificial neural systems, the models are physical entities (neuromimes, simulations of neuromimes, simulations of behavioral models of neuromimes and of elements composed of neuromimes, etc.) rather than tautologies (laws, theorems, etc.), and the fit is a behavioral one, as is every theory, until a new, deeper observation is made of the behavior, or until we are prepared to discard degenerative assumptions that limit our study of currently observed behavior. >The idea seems to be that one can escape the >necessity to achieve an understanding of human perception and leave that >to a machine (or algorithm, rather). Since scientific understanding >(new and old) is so difficult to come by, it's a very seductive idea. >But not a reasonable one. I seem to remember having this same conversation before...anyway: Doing neural nets this way is akin to allowing probability to be a mathematical field, and to statistical mechanics and quantum theory. The understanding has, and consciously so, been behind the techniques in those areas since the techniques were first found to be superior to the understanding in predictive power. --Blair "It's quite reasonable. It's quite reasonable to assume that my thesis won't be half this erudite."
jtn@potomac.ads.com (John T. Nelson) (08/15/89)
> 6. The "popularity" of neural net research is a consequence of the > miserable mathematical backgrounds of computer science students (and > some professors!). You don't need to know any math to be a hacker, but > you have to know math and statistics to work in statistical pattern > recognition. Thus, generations of computer science students are > susceptible to hoodwinking by neat demos based on simple mathematical > and statistical techniques that incorporate some engineering hacks > that can be tweaked forever. They'll think they are accomplishing > something by their endless tweaking because they don't know enough > math and statistics to tell what's really going on. A sweeping generalization. Computer scientists aren't the only ones working on neural networks and not all computer scientists are "student hackers." I wish people would stop confusing "programming" activities with thinking and research activities. They are distinctly different. One is engineering and the other is not. There are computer scientists who approach problems as theoreticians and there are computer scientists who approach problems with ad hoc solutions in mind. However...... (time to get up on my soapbox oh boy!).... In my opinion we don't have a deep macroscopic understanding of what neural nets are capable of doing or are doing even in the simplest networks. Researchers are spending a lot of time and effort focusing on the optimization of small techniques (e.g. backpropigation) and too little time on developing formalisms for describing and understanding NNs as a whole. A deep understanding of any complex paradigm will be reached only through the efforts of many researchers, tackling the problem from different viewpoints (like multiple sculptors chipping away at a block of marble to reveal the statue hidden inside). It's fairly useless for all of these metaphorical artists to chip away at a big toe all at once, yet they must also posses the same overall goal and understanding of the problem, otherwise the final piece will not be consistant and balanced. Well you get the idea.