petersja@debussy.cs.colostate.edu (james peterson) (10/06/90)
Recent postings (taking a slow detour from the original discussion of "emergent properties") have raised the issue of what the paradigmatic "reasoning" might be. Minsky's postings have (not surprisingly) suggested that rule-based, logico-mathematical, reasoning is the basis of all other types of reasoning (or is perhaps the only true reasoning) -- Others have suggested that there are more "natural" types of reasoning which cannot be reduced to inferential logical reasoning (the example given was in the realm of interpersonal decision-making). Minsky, and others, have a lot bet on the idea that *all* mental processes are ultimately (or fundamentally) calculative formal manipulations (though we are of course not always aware that these underlying calculations are taking place), thus formal operations ground *all* thought processes, and therefore, all reasoning, regardless of whether any "reasoning" *seems* non-formal (appearances can be deceiving). This is Smolensky's chicken and egg problem: Does "hard," logical, rule-based reasoning ground all reasoning, or does "soft," evaluative, inductive, rough, pattern recognizing, fuzzy, kinds of reasoning ground all our thought processes, including "hard" scientific thinking? Or are they independent? Smolensky suggests that soft reasoning grounds the hard, while Minsky (and Fodor) appear to believe that hard thinking grounds any soft thinking. I believe many people have a harder time seeing how rigid mathematical thinking could ever be based upon something less rigid, than to conceive of fuzzy reasoning being based upon more rigid underlying principles. Of course, this does not make the easier path so..... I would be interested in any succinct account of how it might be possible for "natural," non-formal, processes (e.g., pattern recognition, or reasoning by analogy, assuming them to be non-formal) to provide the basis for supervenient formal thought. -- james lee peterson petersja@handel.cs.colostate.edu dept. of computer science colorado state university "Some ignorance is invincible." ft. collins, colorado 80523
minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) (10/06/90)
I am absolutely astounded at the statement by petersja@debussy.cs.colostate.edu to that: > Minsky's postings have (not surprisingly) suggested > that rule-based, logico-mathematical, reasoning is the > basis of all other types of reasoning > (or is perhaps the only true reasoning). I hope that no one in net-world will believe that I ever said or maintained anything of the sort. Most of my published work attacks this idea. "The Society of Mind" scarecly mentions logic and rule based reasoning at all, save to place it as among the forms of reasoning used occasionally by people over the age of about 10. Instead, I have maintained from the early 1960s that most thinking uses various forms of pattern-matching and analogy. So I am annoyed at such pseudo-quotes as > Minsky, and others, have a lot bet on the idea that *all* mental processes are ultimately (or fundamentally) calculative formal manipulations (though we are of course not always aware that these underlying calculations are taking place), thus formal operations ground *all* thought processes, and therefore, all reasoning, regardless of whether any "reasoning" *seems* non-formal (appearances can be deceiving). But maybe it is not so strange, come to think of it, that james lee peterson could find such things in "Minsky's postings". You can verify, by running back through the files, that I have said nothing of the sort. However, we could explain this by assuming that Peterson was thinking, by analogy, that because he has heard somewhere that I am a "hard AI" teacher, therefore, I must hold such positions -- and hence, must have said such things. Sorry to waste your time by wordy self-defense, but I'd really like people to read "Society of Mind" and not assume that they can guess what is in it on the basis of hostile stereotype.
loren@tristan.llnl.gov (Loren Petrich) (10/06/90)
In article <3586@media-lab.MEDIA.MIT.EDU> minsky@media-lab.media.mit.edu (Marvin Minsky) writes: > > >I am absolutely astounded at the statement by >petersja@debussy.cs.colostate.edu to that: > >> Minsky's postings have (not surprisingly) suggested >> that rule-based, logico-mathematical, reasoning is the >> basis of all other types of reasoning >> (or is perhaps the only true reasoning). > >I hope that no one in net-world will believe that I ever said or >maintained anything of the sort. Most of my published work attacks >this idea. "The Society of Mind" scarecly mentions logic and rule >based reasoning at all, save to place it as among the forms of >reasoning used occasionally by people over the age of about 10. > >Instead, I have maintained from the early 1960s that most thinking >uses various forms of pattern-matching and analogy... So Marvin Minsky himself has shown up on Internet. I would like to say that I myself have read "Society of Mind" and I think that it is a very interesting proposal on how thinking works, if nothing else. He is probably correct about pattern matching and analogy being a major form of reasoning. But I wonder how much of our reasoning works by what might best be called "fuzzy logic" -- a logic in which predicates can not only have the values "true" or "false", but any value in between. That may well describe how we reason about uncertain things. Traditional logic presupposes a discreteness that is often lacking in the world around us. Any comments? I somehow suspect, however, that certain of Minsky's work may be taken as going in the opposite direction from what he has proposed in "Society of Mind". I mention, in particular, his work with Seymour Papert published in "Perceptrons", published in the late 1960's. At that time, an early Neural Net architecture, the Perceptron, was a very hot topic. The book showed that perceptrons with only one layer of decision units between the inputs and the outputs were severely limited in what they could "perceive" -- that they could only distinguish inputs separated by a hyperplane in input space. This problem could be circumvented by adding extra decision units in between, what are now called "hidden layers", but there seemed to be no way to train such a system. Thus, work on perceptron-like pattern-recognition systems, which are now called Neural Nets, languished for nearly two decades. Since that time, variations on the original perceptron architecture have been discovered, variations that allow straightforward training, with the backpropagation algorithm, for example. Over that last couple of years, interest in NN's has increased explosively. I have read through a number of volumes of conference papers of the theory and practice of NN's. I myself have gotten into work with NN's; I am currently involved in a project to design hardware NN's here at LLNL. Part of this work has involved using NN's to (1) analyze the spectra of plasmas produced in etching chips and (2) construct a function to fit data on thin-film deposition as a function of deposition conditions. In (1), we obtained the somewhat obvious result that, to find the hydrogen content of etching-chamber gas, one must look at hydrogen lines. But we found that the NN looked at at least one CO line also, though it looked at no other lines. In (2), we found that the NN agreed the data as well as a polynomial fit, but we found that the NN outperformed the polynomial fits on data that neither had been trained on. I have also used NN's to classify the sources listed in the IRAS Faint Source Catalog; I found that they fell into two categories, one of sources with little interstellar reddening, and one of sources that are apparently heavily reddened. I feel that there is much more promise in NN's than in traditional AI, which has been dependent on working out decision rules explicitly. Perhaps the most successful application of traditional AI has been computerized algebra, because that is one field where most of the decision rules are known explicitly; many having been known explicity for at least a couple centuries, as a matter of fact. For NN's, however, the "decision rules" are all implicit in the parameter values; a learning algorithm saves us the trouble of having to work them out explicitly. I speak from personal experience, because when I first saw a NN program in action, I was amazed to see that it could actually recognize patterns, a long-time goal of AI that has seemed almost perpetually beyond reach. Also, most AI systems have seemed formidably complex, while NN's are so simple one wonders why the field has not taken off earlier. Compared to a page or two of Fortran or C code for an NN, most AI systems have given the appearance of being elaborate and cumbersome software packages. Contrary to NN's, I have read a lot about traditional AI systems, but I have never actually used one, with the exception of some computer-algebra programs. Not that I do not appreciate the development of computerized algebra; I think that that is an important type of software. So that is why I have been working on projects involving NN's lately. I wonder how Minsky himself would respond to the charges that his work on perceptrons had set the field back for nearly two decades. And I wonder how Minsky feels about NN's themselves. And I suspect that I will be flamed for my assertion that computerized algebra has been about the only big success of traditional AI techniques. I would certainly like to be told about some counterexamples, though. $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Loren Petrich, the Master Blaster: loren@sunlight.llnl.gov Since this nodename is not widely known, you may have to try: loren%sunlight.llnl.gov@star.stanford.edu
minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) (10/06/90)
I agree with most of what loren@tristan.llnl.gov (Loren Petrich) said in article 62. The only problem I have is with his assertion > I feel that there is much more promise in NN's than in traditional > AI, which has been dependent on working out decision rules explicitly. It is not an either-or thing, in my view. NN's are strong in learning to recognize (some) patterns in which something depends on many other things in relatively weak dependencies. NN's can represent such relationships when they have good linear approximations -- but, probably, only in those domains. We don't know a lot about how to characterize them. But lots of human pattern recognition machinery probably uses this. On the other side, the PROCEDURES that can be represented in NN's are very limited, certainly in the non-cyclic nets that dominate the work of the 80s. This means that, without a lot of external script-like control, it will be hard for them to reason about what they have recognized. A careful re-reading of "Perceptrons" will show that virtually all the negative results therein still hold for multi-layer noncyclic networks -- especially theoriems like the AND-OR theorem which show why an NN that recognizes parts may not be able to (learn to) recognize when those parts have particular relationships, etc. I could go on about this, but the point is this: 1. Yes: systems with compact rules with very few input terms are not good at recognizing patterns which need many inputs. So AI systems restricted to compact rules must be supplemented by NN-like structures. 2. No: the NN-like structures cannot replace the "reasoning systems" of "traditional AI", unless we supply architectures that embody those goal-oriented processes. For example, "annealing" does not replace all other kinds of intelligent heuristic search. A tricky fallacy is to think, "Golly, I have now seen NN's solve a hundred problems in the last five years that 'old AI' couldn't solve. What's wrong with that is (i) you can look at it the other way: let's see NNs learn to solve formal integration problems, or similar problems that involve dissection of descriptions and (ii) many of those problems NNs can solve can also be solved by other kinds of analysis -- and, sometimes in ways that lend themselves to being usable in OTHER situations. In this sense, then, NN solutions, in contrast, tend to be dead ends, simply because what you end up with, after your 100,000 steps of hill-climbing, is an opaque vector of coefficients. You have solved the prob lem, all right. You have even _learned_ the solution! But you don't end up with anything you can THINK about! Is that bad? Your locomotion system "learns" to walk, all right. (It begins with an architecture of NN's that wonderfully work to adjust your reflexes.) But "you" don't know anything of how it's done. Even Professors of Locomotion Science are still working out theories about such things. So may you can make a pretty good dog with NNs. And note that I put NNs in the plural! A dog, or a human, learns by using a brain that consists of (I estimate) some 400 clearly distinctly different NN architectures and perhaps 3000 distinct busses or bundles of specialized interconnections. What does that mean? Answer: some of the job is done by NNs. And some of the job is done by compactly-describable procedural specifications. Where is the "traditional, symbolic, AI in the brain"? The answer seems to have escaped almost everyone on both sides of this great and spurious controversy! The 'traditional AI' lies in the genetic specifications of those functional interconnections: the bus layout of the relations between the low-level networks. A large, perhaps messy software is there before your eyes, hiding in the gross anatomy. Some 3000 "rules" about which sub-NN's should do what, and under which conditions, as dictated by the results of computations done in other NNs (see the idea of "B-brain" in my book). Someone might object that this may be an accident. In a few years, perhaps, someone will find a new learning algorithm through which a single, homogeneous NN (highly cyclic, of course) can start from nothing and learn to become very smart, without any of that higher-level stuff encoded into its anatomy -- and all in some reasonable amount of time. That is the question, and I see no reason to think that present-day results are very encouraging. ----- Here is a simple, if abstract, example of what I mean. Consider one of the most powerful ideas in traditional AI -- the concept of acheiving a goal by detecting differences between the present situation ("what you have") and a target situation ("what you want"). The Newell and Simon 'GPS' system did such things (and worked in many cases, but not all) by trying various experiments and comparing the results, and then applying strategies designed (or learned) for 'reducing' those differences. In order to do this, common sense would suggest, you need resources for storing away the various recent results, and then pulling them out for comparisons. This is easily done with the equivalent of registers, or short-term memories -- and it seems -- from a behavioral viewpoint -- that human brains are equipped with modest numbers of such structures. Now, in fact, no one knows the physiology of this. In "Society of Mind" I conjecture that many of our brain NN's are especially equipped with what I call "temporary K-lines" or "pronomes" that are used for such purposes. (Their activities are controlled by other NN's that somehow learn new control-scripts for managing those short-term memories.) Well, if you design NNs with such facilities, then it will not be very hard to get them to solve symbolic, analytic problems. If you don't provide them with that sort of hardware, everything will get too muddled, and (I predict) they'll "never" get very far. It will be like trying to teach your dog to do calculus. An alternative will be to design a fiendishly clever pre-training scheme which "teaches" your NN, first, to build inside itself some registers. This might indeed be feasible, with a homogeneous NN, under certain conditions. But it wouldn't be exactly a refutation of what I said before, because it would involve, not the NN itself "discovering" an adequate architecture, but an external teacher's deliberately imposing that architecture on the NNs future development. (Even this is not all-or-none, because there is clearly some such trade-off in human development which, according to all accounts, will fail in the absence of any attentive adult caretaker. Oh well. ---------- In any case, I want to thank Loren for endless thoughtful observations about many other topics. I intend to think more about what he said here.
dave@cogsci.indiana.edu (David Chalmers) (10/06/90)
In article <3593@media-lab.MEDIA.MIT.EDU> minsky@media-lab.media.mit.edu (Marvin Minsky) writes: >In this sense, then, NN solutions, in >contrast, tend to be dead ends, simply because what you end >up with, after your 100,000 steps of hill-climbing, is an opaque >vector of coefficients. You have solved the prob lem, all right. You >have even _learned_ the solution! But you don't end up with anything >you can THINK about! >Is that bad? Your locomotion system "learns" to walk, all right. (It >begins with an architecture of NN's that wonderfully work to adjust >your reflexes.) But "you" don't know anything of how it's done. Even >Professors of Locomotion Science are still working out theories about >such things. I hear this kind of thing said often enough, but I don't buy it. Sure, producing a computational system that does something doesn't immediately *explain* how something is done, but it certainly makes explanation a lot easier. The "brains are right in front of us, but we still don't understand them" argument doesn't really hold water. Most of the problems with brains are problems of *access* -- they're nasty and gooey and people tend to complain if you poke around and cut them up too much. Current neuroscience is mostly constrained by technological limitations. To see this, witness the huge flurry of activity that takes place whenever a new tool for brain investigation -- PET scanning, for instance -- is devised. Whereas if we produce an equivalent computational system, all those problems of access are gone. We have the system right in front of us, we can poke around its insides and make complex observations to our heart's content. We can perform fast and easy simulations of its function in all kinds of environments. We can lesion this, monitor that, investigate the consequences of all manner of counterfactual situations -- all without running into trouble with blood and goo or ethics committees. If the Professors of Locomotion Science had a perfect computational model of the locomotive system in front of them, you can bet that progress in the area would proceed one hundred times faster. If I had "the program" of the brain stored in a file on my Sun workstation, within five years cognitive science would be completely transformed. We probably wouldn't understand *everything* about language and learning and memory, but we would understand a hell of a lot more than we do now. So, a computational model of a system is not equivalent to an explanation of the system. But once you have the model, an explanation may not be far away. -- Dave Chalmers (dave@cogsci.indiana.edu) Concepts and Cognition, Indiana University. "It is not the least charm of a theory that it is refutable."
loren@tristan.llnl.gov (Loren Petrich) (10/06/90)
In article <3593@media-lab.MEDIA.MIT.EDU> minsky@media-lab.media.mit.edu (Marvin Minsky) writes: > > >I agree with most of what loren@tristan.llnl.gov (Loren Petrich) said >in article 62. The only problem I have is with his assertion > >> I feel that there is much more promise in NN's than in traditional >> AI, which has been dependent on working out decision rules explicitly. You're right. I goofed. I concede that there are things that traditional AI techniques can do better than most NN's. I doubt that NN's will ever pose much competition in fields like computer algebra, where most of the inference rules are straightforward and unambiguous, and have been well understood for a long time. There are other difficulties with NN's, at least at the present time. For instance, NN's are generally constructed around data structures that are linear and whose lengths are fixed. This is OK for a wide range of problems, but there are difficulties for representing data structures whose length may vary, and even which are nonlinear, an example being a treelike one. There are tricks I have seen for getting around that, but even there, a NN will probably have to be "managed" by some outside system. But my point was, why attempt to painstakingly work out hundreds of complicated and imprecise inference rules when the whole job can be done automatically? > 1. Yes: systems with compact rules with very few input terms are not >good at recognizing patterns which need many inputs. So AI systems >restricted to compact rules must be supplemented by NN-like >structures. > 2. No: the NN-like structures cannot replace the "reasoning >systems" of "traditional AI", unless we supply architectures that >embody those goal-oriented processes. For example, "annealing" does >not replace all other kinds of intelligent heuristic search. I agree. > ... In this sense, then, NN solutions, in >contrast, tend to be dead ends, simply because what you end >up with, after your 100,000 steps of hill-climbing, is an opaque >vector of coefficients. You have solved the prob lem, all right. You >have even _learned_ the solution! But you don't end up with anything >you can THINK about! I understand your point. However, my colleagues and I have occasionally been able to interpret the weight values produced by NN's. One project we did was to evaluate spectra produced in etching chips. By examining them, we hoped to train a NN to determine how much hydrogen was in the etching chamber. We discovered that the weights were largest in some small regions of the spectrum. These corresponded to lines of H and one of CO, a reaction product. It was surprising to us that the NN might have been using a CO line as a diagnostic for the amount of hydrogen. An improved version might be set up to look only at H and CO lines, given what the first one ended up focusing on. I think that the difficulty of not learning too much about what one wants to recognize is far from fatal in practice, however desirable in theory may be. >Here is a simple, if abstract, example of what I mean. Consider one >of the most powerful ideas in traditional AI -- the concept of >acheiving a goal by detecting differences between the present >situation ("what you have") and a target situation ("what you want"). >The Newell and Simon 'GPS' system did such things (and worked in many >cases, but not all) by trying various experiments and comparing the >results, and then applying strategies designed (or learned) for >'reducing' those differences. I see the point. It seems to me very difficult to imagine how to get a NN to do something like that. >In any case, I want to thank Loren for endless thoughtful observations >about many other topics. I intend to think more about what he said here. Thank you. I have been interested in AI and I have become rather disappointed at its slow rate of progress over the years. However, it is good to know that now we can make progress somewhere. $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Loren Petrich, the Master Blaster: loren@sunlight.llnl.gov Since this nodename is not widely known, you may have to try: loren%sunlight.llnl.gov@star.stanford.edu
nagle@well.sf.ca.us (John Nagle) (10/07/90)
minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) writes: >Is that bad? Your locomotion system "learns" to walk, all right. (It >begins with an architecture of NN's that wonderfully work to adjust >your reflexes.) It is not clear that walking has to be learned. The fact that horses can stand within an hour of birth and run with the herd within a day suggests otherwise. The human developmental sequence may be misleading here, humans being born in a less complete state than some of the lower mammals. >So may you can make a pretty good dog with NNs. In our present state of ignorance, we would have difficulty making a good ant with NNs. The work of Rod Brooks and Patty Maes at MIT shows that some simple locomotion problems can be dealt with in that way, but full ant functionality has not been achieved. Beer, Chiel, and Sterling at CWRU are further along toward full insect functionality, and, interestingly enough, their model of neural net components resembles more closely the observed biological data, rather than following the connectionist backward propagation approach. If you believe Sir John Eccles, all the mammals have roughly the same brain architecture and the differences between the various mammmals are quantitative, not qualitative. Dissection, DNA distance, and the evolutionary timetable all point in that direction. So if we can make it to dog-level AI, we should be almost there. But we aren't even close. John Nagle
minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) (10/07/90)
In article <21054@well.sf.ca.us> nagle@well.sf.ca.us (John Nagle) writes: >minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) writes: > >>Is that bad? Your locomotion system "learns" to walk, all right. (It >>begins with an architecture of NN's that wonderfully work to adjust >>your reflexes.) > > It is not clear that walking has to be learned. The fact that >horses can stand within an hour of birth and run with the herd within >a day suggests otherwise. The human developmental sequence may be >misleading here, humans being born in a less complete state than some >of the lower mammals. I agree. That's why I said "learned" instead of learned. But the point is that there remains a powerful "tuning-up" process that takes only a modest number of minutes to get close and only a few hours to get pretty good. But my point -- and Nagle's, too -- is that we're born with pretty much the right network, in which the relevant inputs are genetically brought close to where they need to be, so that the naimal does not have to explore a huge space and be in danger of getting trapped in bad, but locally optimal, configurations. > If you believe Sir John Eccles, all the mammals have roughly >the same brain architecture and the differences between the various >mammmals are quantitative, not qualitative. Dissection, DNA distance, >and the evolutionary timetable all point in that direction. So if we >can make it to dog-level AI, we should be almost there. But we aren't >even close. Probably not. We'll probably discover a small number of small but qualitatively critical differences, e.g., in the organization of short-term memories, maintanence of small but important sub-goal trees, and a few other sorts of AI-type resources. Yes, comparative anatomists will continue to say that these differences are small. But as we all know, 10 is almost 11, and 11 is almost 12, etc. What I mean is that Eccles is surely right, in that our huge forebrain doesn't seem very different from earlier, smaller forebrains. But I'll bet he'll turn out wrong on some small-scale functional level. Something probably happened 5 million years ago to make all that additional machinery useful -- instead of a handicap. Some small but ciritical change on the "management" level.
rolandi@sparc9.hri.com (Walter Rolandi) (10/07/90)
In article <21054@well.sf.ca.us>, nagle@well.sf.ca.us (John Nagle) writes: > minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) writes: > > >So may you can make a pretty good dog with NNs. > > If you believe Sir John Eccles, all the mammals have roughly > the same brain architecture and the differences between the various > mammmals are quantitative, not qualitative. Dissection, DNA distance, > and the evolutionary timetable all point in that direction. So if we > can make it to dog-level AI, we should be almost there. But we aren't > even close. > > John Nagle I too was surprized at this statement. It is astounding how little AI people seem to know about biological learning. One can establish a generalized same-different discrimination in a dog in at most, a few weeks. I don't see this happening anytime soon with machines. You know what you AI philosophers should do? Go out and buy yourself a puppy. Get yourself a good book on dog training (best ever written was by a guy named Daniel Tortora) and personally put your dog through obedience training---never using any punishment or negative reinforcement. You will learn a great deal about learning in the course of discovering how little you know. -- ------------------------------------------------------------------------------ Walter G. Rolandi Horizon Research, Inc. 1432 Main Street Waltham, MA 02154 USA (617) 466 8367 rolandi@hri.com ------------------------------------------------------------------------------
jmc@Gang-of-Four.usenet (John McCarthy) (10/08/90)
Peters writes "This is Smolensky's chicken and egg problem: Does "hard," logical, rule-based reasoning ground all reasoning, or does "soft," evaluative, inductive, rough, pattern recognizing, fuzzy, kinds of reasoning ground all our thought processes, including "hard" scientific thinking? Or are they independent? Smolensky suggests that soft reasoning grounds the hard, while Minsky (and Fodor) appear to believe that hard thinking grounds any soft thinking." Minsky will doubtless tell you that logic isn't what he is enthusiastic about, whereas I am an enthusiast for logic. However, I wouldn't claim that logic "grounds" all reasoning, because I think grounding is an oversimplified notion. The human ability to do logic developed from and still uses processes that can be called reasoning but don't correspond to logic. These processes are inaccurate in unnecessary and inconvenient ways. These inaccurate human processes did form a desire to develop accurate reasoning processes, i.e. logic. As a branch of mathematics, logic is grounded in formal semantics as Tarski and others have described, i.e. it has been made independent of the thought processes that motivated its development. For full AI, mathematical logic needs supplements such as formalized nonmonotonic reasoning and probably formalized contexts, but these aren't reversions to ordinary soft thinking. We can make an analogy with the fact that we can write an interpreter for any good programming language in any another. We can talk about logic in ordinary language, and we can formalize ordinary language and reasoning in logic.
csma@lifia.imag.fr (Ch. de Sainte Marie) (10/09/90)
In article <69347@lll-winken.LLNL.GOV> loren@tristan.llnl.gov (Loren Petrich) writes: > [...] >a major form of reasoning. But I wonder how much of our reasoning >works by what might best be called "fuzzy logic" -- a logic in which >predicates can not only have the values "true" or "false", but any >value in between. That may well describe how we reason about uncertain >things. Traditional logic presupposes a discreteness that is often >lacking in the world around us. Any comments? It seems to me that reasonning is, to a great extent, about making choices; fuzzy logic is all about avoiding choices as long as possible. So, fuzzy logic does not seem like the best tool for rational thinking. I agree (just because it would'nt change the condition if I did'nt :-) that factual knowledge about the world around us most often lacks discreteness, but I propose (and I suppose as a working hypothesis) that a prerequisite to reasonning is forcing discreteness on reluctant data. In hypothetico-deductive reasonning, you don't use fuzzy logic (I, for one, don't: I could'nt possibly, I don't even know what it is): you examine what would be the outcome, should this or that be true (or false). It seems to me that when one reasons in face of uncertainty, one proceeds in a similar way: choose some facts to be true, others to be false, and throw out the rest as irrelevant; then, use the newly `certain' facts to reason. The criterion by which one selects which facts to use, and which to reject, can be (in facts, is most probably) probabilist to an extent, but the logic does not seem to be the least fuzzy. One can eventually have to change what one chose to believe, but it can only be when faced with more information, and then one just begins all the process anew. The argument is introspective, and thus suspect, but I could'nt think of a better (short) one at this time of day... Well, it's all a bit simplistic (and I'm a bit clumsy after a long day working, and so are my comments -but not my position, which is very clear. To me:-), but it seems essentially sound to me; so, I'll stick with good ol' bivalued (or trivalued) logic. Or does somebody have a good argument against that position? -- Ch. de Sainte Marie - LIFIA - 40, av. Felix Viallet - 38031 Grenoble - FRANCE csma@lifia.imag.fr csma@lifia.UUCP "C'est ephemere et c'est moi..." {uunet.uu.net|mcvax|inria}!imag!lifia!csma
sarima@tdatirv.UUCP (Stanley Friesen) (10/09/90)
In article <69377@lll-winken.LLNL.GOV> loren@tristan.llnl.gov (Loren Petrich) writes: >In article <3593@media-lab.MEDIA.MIT.EDU> minsky@media-lab.media.mit.edu (Marvin Minsky) writes: > There are other difficulties with NN's, at least at the >present time. For instance, NN's are generally constructed around data >structures that are linear and whose lengths are fixed. This is OK for >a wide range of problems, but there are difficulties for representing >data structures whose length may vary, and even which are nonlinear, >an example being a treelike one. I suspect that this is more an indication of the relative immaturity of NN technology, since human brains seem to be able to deal with non-linear, variable-sized data structures reasonably well. [e.g. human language]. I have no more idea than you what the solution is in NN technology, but I suspect it will be found. >> 2. No: the NN-like structures cannot replace the "reasoning >>systems" of "traditional AI", unless we supply architectures that >>embody those goal-oriented processes. For example, "annealing" does >>not replace all other kinds of intelligent heuristic search. > I agree. Except that, again, I suspect many of the current limitations in NN's will disappear with time. Certainly any useful heuristic technique which humans are capable of can be implemented in NN's. So unless current AI software is using non-human heuristics, there is no long-term barrier to NN's replacing traditional AI in this area also. [And even if non-human heuristics are being used it may be that the human ones are in some sense better any way]. The main reason I see for continuing to use traditional AI techniques is the question of efficiency. For certain classes of heuristics and decision processes traditional programming may produce a faster and/or cheaper implementation. > I understand your point. However, my colleagues and I have >occasionally been able to interpret the weight values produced by >NN's. One project we did was to evaluate spectra produced in etching >chips. By examining them, we hoped to train a NN to determine how much >hydrogen was in the etching chamber. We discovered that the weights >were largest in some small regions of the spectrum. These corresponded >to lines of H and one of CO, a reaction product. It was surprising to >us that the NN might have been using a CO line as a diagnostic for the >amount of hydrogen. Good example. I suspect this type of result may prove to be common. One avenue towards more advanced AI systems might be to try to automate this process of meaning extraction from the NN weight values. If we xould do this we would have gone a long way towards developing a self-learning system. It would also provide a solid basis for conceptual analysis. >>Here is a simple, if abstract, example of what I mean. Consider one >>of the most powerful ideas in traditional AI -- the concept of >>acheiving a goal by detecting differences between the present >>situation ("what you have") and a target situation ("what you want"). >>The Newell and Simon 'GPS' system did such things (and worked in many >>cases, but not all) by trying various experiments and comparing the >>results, and then applying strategies designed (or learned) for >>'reducing' those differences. > > I see the point. It seems to me very difficult to imagine how >to get a NN to do something like that. Perhaps so. But again this seems to be fairly close to how humans approach difficult goals without an obvious solution. And since the human brain is, by definition, an NN this consititutes an existance proof for a way of solving this problem in NN's. [The only way to counter this is to provide evidence from psychology that humans do not, in fact, ever solve problems this way] -- --------------- uunet!tdatirv!sarima (Stanley Friesen)
sticklen@cps.msu.edu (Jon Sticklen) (10/09/90)
From article <JMC.90Oct8083235@Gang-of-Four.usenet>, by jmc@Gang-of-Four.usenet (John McCarthy): > ... > We can make an analogy with the fact that we can write an interpreter > for any good programming language in any another. We can talk about > logic in ordinary language, and we can formalize ordinary language and > reasoning in logic. Although we can write an interpreter for any good programming language in any other, the more salient obsevation is that certain operations are more easily carried out in particular langauges. Eg, if I want to maniupulate lists, I am a bit more likely to want to use LISP than I am to want to use COBOL, although each is a general purpose language. Even more central is the example that should I want to do a statistics problem of some sort, I would be yet more likely to select SPSS as my language of choice. The reason would seem clear - SPSS gives me exactly the language I want to describe the statistics problem at hand. The generalization of this line - ie, special purpose languages allow easier represtatation of problem solving situations *they fit* - leads in knowledge based systems to the Task Specific Arch schools that have developed over the last decade. (Eg, Chandrasekaran, McDermott, Steels, etc) The bottom line may be that although we *could* represent problem solving with general purpose tools, that it may be much easier to do with a suite of techniques/tools each tuned for their particular problem solving situations. ---jon---
ghh@clarity.Princeton.EDU (Gilbert Harman) (10/09/90)
In article <JMC.90Oct8083235@Gang-of-Four.usenet> jmc@Gang-of-Four.usenet (John McCarthy) writes: Minsky will doubtless tell you that logic isn't what he is enthusiastic about, whereas I am an enthusiast for logic. However, I wouldn't claim that logic "grounds" all reasoning, because I think grounding is an oversimplified notion. The human ability to do logic developed from and still uses processes that can be called reasoning but don't correspond to logic. These processes are inaccurate in unnecessary and inconvenient ways. These inaccurate human processes did form a desire to develop accurate reasoning processes, i.e. logic. As a branch of mathematics, logic is grounded in formal semantics as Tarski and others have described, i.e. it has been made independent of the thought processes that motivated its development. For full AI, mathematical logic needs supplements such as formalized nonmonotonic reasoning and probably formalized contexts, but these aren't reversions to ordinary soft thinking... It is important to distinguish the theory of implication (logic) from the theory of what you have reasons to conclude given the beliefs (and plans and goals) that you start with (the theory of reasoning). The theory of logic has nothing special to say about what reasoning a person ought to do (or is permitted to do). Logical theory does not make use of normative notions like "ought" or "permission". Nor does logical theory have a psychological subject matter. So logic is not about what to infer, if inference is a matter of arriving at certain beliefs. There are special deontic logics concerning the implications of propositions about what someone ought to do and there are special logics that concern the implications of propositions about beliefs. But these logics are themselves concerned with implications of certain propositions and not with what one should infer (i.e. believe) about these subjects. It is clear, of course, that people reason about implications. People reason when doing logic. But it is (or should be) equally clear that the principles of logic are not principles that people do or ought to follow in their reasoning. The principles of logic are simply not about what conclusions can be reached under certain conditions. Anyone tempted to suppose that there is a fairly direct connection between logic and some sort of reasoning might look at my discussion of this issue in CHANGE IN VIEW (MIT, 1986), chapters 1 and 2. I think I'm agreeing with Minsky and disagreeing with McCarthy. -- Gilbert Harman Princeton University Cognitive Science Laboratory 221 Nassau Street, Princeton, NJ 08542 ghh@clarity.princeton.edu HARMAN@PUCC.BITNET
jacob@latcs1.oz.au (Jacob L. Cybulski) (10/09/90)
From article <3586@media-lab.MEDIA.MIT.EDU>, by minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky): >> Minsky's postings have (not surprisingly) suggested >> that rule-based, logico-mathematical, reasoning is the >> basis of all other types of reasoning > > I hope that no one in net-world will believe that I ever said or > maintained anything of the sort. Most of my published work attacks > this idea. To further this statement, here in Australia, Minsky is regarded as one of the greatest "scruffies" of Artificial Intelligence and Cognitive Science, as opposed to logico-oriented "neaties". Jacob L. Cybulski Amdahl Australian Intelligent Tools Program Department of Computer Science La Trobe University Bundoora, Vic 3083, Australia Phone: +613 479 1270 Fax: +613 470 4915 Telex: AA 33143 EMail: jacob@latcs1.oz
jhess@orion.oac.uci.edu (James Hess) (10/09/90)
Let me take a first stab at James Peterson's question, which I will roughly paraphrase as 'how can pattern recognition and similar fuzzy processes form the basis for a formal symbolic reasoning system". I suggest that this is a case of complexity, in a formal sense. We do not have one process or the other as a primitive in the mind out of which the other is constructed and to which it is therefore reducible; both play a vital and necessarily vital role. For example, it seems awkward to reduce "you and I will go to the store, but only if it is not raining" to pattern recognition, especially if we look at the similarity between that sentance and "you and I will go to the store, but only if it is not after ten o'clock". How could pattern recognition devoid of symbolic manipulation model the similarity of "after ten o'clock" and "raining" in order to recognize that they both make sense in that context, although one is an external physical phenomena and the other an abstract mental construction? On the other hand, how are we to recognize a new example of a chair? If we were to list the properties of a chair, in almost every case we could find a chair that lacks one of them or a non-chair that possesses several or many of them. We cannot refer to a set of necessary and sufficient conditions. We may get caught in the trap of infinite regress if we try to specify all the general rules and exception rules that define an object as a member of a class A. Do people really go through this list when they make classifications? Or is it a fuzzy, statistical pattern recognition process? Melville went to some length in the debate over whether a whale was a mammal or a fish. The perceptual and behavioral qualties are ambiguous; only later did a whale become a mammal by fiat.
petersja@debussy.cs.colostate.edu (james peterson) (10/09/90)
In article <JMC.90Oct8083235@Gang-of-Four.usenet> jmc@Gang-of-Four.usenet (John McCarthy) writes: >grounding is an oversimplified notion. The human ability to do logic >developed from and still uses processes that can be called reasoning >but don't correspond to logic. These processes are inaccurate in >unnecessary and inconvenient ways. These inaccurate human processes did >form a desire to develop accurate reasoning processes, i.e. logic. What I am interested in is the nature of these "processes that can be called reasoning but don't correspond to logic" -- When you say "don't correspond" are you implying that they are irreducible to logical reasoning (I suspect you aren't)? And these "inaccurate processes," are they actually instantiations of formal rule manipulation at a lower level of description? Put another way, are they Turing computable? > >We can make an analogy with the fact that we can write an interpreter >for any good programming language in any another. We can talk about >logic in ordinary language, and we can formalize ordinary language and >reasoning in logic. In "formalizing ordinary language" is there a residue that escapes formalization? -- james lee peterson petersja@handel.cs.colostate.edu dept. of computer science colorado state university "Some ignorance is invincible." ft. collins, colorado 80523
minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) (10/09/90)
In article <11@tdatirv.UUCP> sarima@tdatirv.UUCP (Stanley Friesen) writes: >..... And since the human brain is, >by definition, an NN this consititutes an existance proof for a way of >solving this problem in NN's. This is a dangerous rhetorical trick. Because in the usual context of discussion, a "neural network" or NN is considered to be a relatively homogeneous, uniform structure equipped with a relatively systematic learning procedure. The brain is at least 400 different architectures interconnected in accord with genetic specifications that appear to involve the order of at least 30,000 genes. So the performance of brain-NNs does not constitute an existence proof for ways to solve similar problems by homogeneous NNs.
sarima@tdatirv.UUCP (Stanley Friesen) (10/09/90)
In article <JMC.90Oct8083235@Gang-of-Four.usenet> jmc@Gang-of-Four.usenet (John McCarthy) writes: >Peters writes > >"This is Smolensky's chicken and egg problem: Does "hard," logical, >rule-based reasoning ground all reasoning, or does "soft," evaluative, >inductive, rough, pattern recognizing, fuzzy, kinds of reasoning >ground all our thought processes, including "hard" scientific >thinking? Or are they independent? Smolensky suggests that soft >reasoning grounds the hard, while Minsky (and Fodor) appear to believe >that hard thinking grounds any soft thinking." If Minsky thinks this he is full of BS! The basic components of fuzzy, soft reasoning are wired into the very fabric of our brain - that is how neural networks work! [And our brain is an NN par excellence]. At most the hard and soft resoning systems in our minds are independent. But I see little evidence for this, humans that have not recieved intensive training in logic or other forms of hard reasoning seem totally incapable of it. We are a poor sample, since by virtue of being programmers we have all recieved much training in hard logic. [Often indirectly, through experimentation and playing, but still training]. -- --------------- uunet!tdatirv!sarima (Stanley Friesen)
sarima@tdatirv.UUCP (Stanley Friesen) (10/09/90)
In article <27116E2C.11522@orion.oac.uci.edu> jhess@orion.oac.uci.edu (James Hess) writes: >For example, it seems awkward to reduce "you and I will go to the store, but >only if it is not raining" to pattern recognition, especially if we look at the >similarity between that sentance and "you and I will go to the store, but only >if it is not after ten o'clock". How could pattern recognition devoid of >symbolic manipulation model the similarity of "after ten o'clock" and >"raining" in order to recognize that they both make sense in that context, >although one is an external physical phenomena and the other an abstract mental >construction? Except that, as near as I can tell, the source of what we call 'symbols' or 'concepts' in the human brain is a sort of subliminal pattern recognition! In the brain a 'symbol' or 'concept' is an association between a representation and a set of patterns, where the association itself is based on a pattern of co-occurance between the representation and its denotation. The tendency to talk about time as if it were a place is apparently based on the recognition of similarities (a pattern) between time and place. L I N E C N T -- --------------- uunet!tdatirv!sarima (Stanley Friesen)
gessel@cs.swarthmore.edu (Daniel Mark Gessel) (10/09/90)
In <11@tdatirv.UUCP> sarima@tdatirv.UUCP (Stanley Friesen) writes: <stuff deleted> >I suspect that this is more an indication of the relative immaturity of NN >technology, since human brains seem to be able to deal with non-linear, >variable-sized data structures reasonably well. [e.g. human language]. >I have no more idea than you what the solution is in NN technology, but I >suspect it will be found. <more stuff deleted> You seem to be starting from the point of view that NNs embody the way the human brain works. Unless there have been some huge jumps in the understanding of the human brain and it's functioning, no one knows if NNs are even close. How the exact chemical composition of the brain affects the functioning of the brain is unknown. Todays artificial NN cannot be assumed to capture this accurately. To assume that future NN's will be able to is to assume that we will be able to recreate the human brain via well defined functions (in the Mathematical sense) which is not necessarily possible. Dan -- Internet: gessel@cs.swarthmore.edu UUCP: {bpa,cbmvax}!swatsun!gessel
jmc@Gang-of-Four.usenet (John McCarthy) (10/10/90)
In article <10081@ccncsu.ColoState.EDU> petersja@debussy.cs.colostate.edu (james peterson) writes: Path: neon!shelby!riacs!agate!apple!julius.cs.uiuc.edu!zaphod.mps.ohio-state.edu!ncar!boulder!ccncsu!debussy.cs.colostate.edu!petersja From: petersja@debussy.cs.colostate.edu (james peterson) Newsgroups: comp.ai.philosophy Date: 9 Oct 90 14:39:55 GMT References: <9963@ccncsu.ColoState.EDU> <JMC.90Oct8083235@Gang-of-Four.usenet> Sender: news@ccncsu.ColoState.EDU Organization: Colorado State Computer Science Department Lines: 32 In article <JMC.90Oct8083235@Gang-of-Four.usenet> jmc@Gang-of-Four.usenet (John McCarthy) writes: >grounding is an oversimplified notion. The human ability to do logic >developed from and still uses processes that can be called reasoning >but don't correspond to logic. These processes are inaccurate in >unnecessary and inconvenient ways. These inaccurate human processes did >form a desire to develop accurate reasoning processes, i.e. logic. What I am interested in is the nature of these "processes that can be called reasoning but don't correspond to logic" -- When you say "don't correspond" are you implying that they are irreducible to logical reasoning (I suspect you aren't)? And these "inaccurate processes," are they actually instantiations of formal rule manipulation at a lower level of description? Put another way, are they Turing computable? > >We can make an analogy with the fact that we can write an interpreter >for any good programming language in any another. We can talk about >logic in ordinary language, and we can formalize ordinary language and >reasoning in logic. In "formalizing ordinary language" is there a residue that escapes formalization? I don't understand human reasoning very well and neither (I think) does anyone else. However, there is no evidence that the basic human reasoning processes correspond to what humans invented (i.e. logic) in our attempt to get a more rational and explicit reasoning process. When we understand them better, I believe human reasoning processes will be formalizable in logic at one remove, e.g. there might be a formula human-believes(person,p) & human-believes(person,implies(p,q)) & foo(person,p,q) => human-believes(person,q). This asserts that humans do modus ponens with a qualification expressed by foo(person,p,q).
nagle@well.sf.ca.us (John Nagle) (10/11/90)
minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) writes: >In article <11@tdatirv.UUCP> sarima@tdatirv.UUCP (Stanley Friesen) writes: >>..... And since the human brain is, >>by definition, an NN this consititutes an existance proof for a way of >>solving this problem in NN's. >This is a dangerous rhetorical trick. Because in the usual context of >discussion, a "neural network" or NN is considered to be a relatively >homogeneous, uniform structure equipped with a relatively systematic >learning procedure. The brain is at least 400 different architectures >interconnected in accord with genetic specifications that appear to >involve the order of at least 30,000 genes. Yes. See section 6.3 of "The Metaphorical Brain 2" (Arbib, M., 1989, Wiley, ISBN 0-471-09853-1), where some wiring diagrams for parts of the nervous system are given. Some hard data in this area is starting to come in. The parts of the nervous system we understand consist of functional units that do special-purpose processing, interconnected in generally plausible ways from a control-theory perspective. There is clear higher-level hard-wired structure in the nervous system. This is not speculation; the experimental results are in. It's also worth bearing in mind that nothing like the backward- propagation learning of the NN world has yet been discovered in biology. The mechanism found so far look much more like collections of adaptive controllers operating control loops. However, it should be noted that most of the neural structures actually understood are either in very simple animals (like slugs) or very close to sensors and actuators (as in tactile control), where one would expect structures that work like adaptive feedback control systems. The more abstract levels of brain processing are still weakly understood. Personally, I favor the hypothesis that the higher levels of processing are built out of roughly the same components as the lower levels. In the absence of experimental data to the contrary, this is a plausible assumption. This leads one to consider some rather different lines of approach than the more popular ones. The various artificial insect groups are proceeding in directions consistent with this assumption. How far it can be pushed remains to be seen. John Nagle
sarima@tdatirv.UUCP (Stanley Friesen) (10/12/90)
In article <3642@media-lab.MEDIA.MIT.EDU> minsky@media-lab.media.mit.edu (Marvin Minsky) writes: >In article <11@tdatirv.UUCP> I write: >>..... And since the human brain is, >>by definition, an NN this consititutes an existance proof for a way of >>solving this problem in NN's. >This is a dangerous rhetorical trick. Because in the usual context of >discussion, a "neural network" or NN is considered to be a relatively >homogeneous, uniform structure equipped with a relatively systematic >learning procedure. The brain is at least 400 different architectures >interconnected in accord with genetic specifications that appear to >involve the order of at least 30,000 genes. Okay, I will back off a little here. Not having followed the NN literature I was unaware of how specialized the definition of NN had become. I had assumed the term was a general one for any parallel network of neuron-like elements. With the revised definition of NN, the human (=mammalian) brain appears to be a three level compound NN (a network of networks of NN's). In this model each functional column in the cerebral cortex would be an NN. These are organized into architectural areas as a diffuse network. Then these areas are organized into a sparsely connected network using functional principles. >So the performance of brain-NNs does not constitute an existence proof >for ways to solve similar problems by homogeneous NNs. Quit true. It does however consist of an existance proof for the use of NN type technology to solve these problems. And if homogeneous NN's cannot do the job a switch to a functional network of NN's very well might. [Indeed a sufficiently general meta-network should be able to solve all of the problems discussed previously] My main point was that there is no reason to assume that complex networks of neuron-like elements have any particular limitations, and that indeed we have a counter-example to most proposed limitations. By the way, given the rather specialized definition of NN, is there an accepted term for a compound NN network? -- --------------- uunet!tdatirv!sarima (Stanley Friesen)
minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) (10/12/90)
In article <22@tdatirv.UUCP> sarima@tdatirv.UUCP (Stanley Friesen) writes: >given the rather specialized definition of NN, is there an accepted >term for a compound NN network? and he makes a number of points about the nervous system. As he suggests, there is more homogeneity at smaller size scales, such as that of cortical columns in the same functional region, than in the gross anatomy of larger scales. As for terminology, I don't think the NN-ers have a language for compound architectures. And surely, without terms for such things, it will be hard for them to get unmuddled. In neurology, they talk about the variety of "cytoarchitectures" in different parts of the brain. But in present-day NN jargon, the researchers generally assume multi-layer perceptrons as the default architecture: that is, stratified layers in which signals go only from input to output. "The Society of Mind" proposes a few terms for describing compound architectures. Not a general, systematic, comprehensive langauge, however, because I tailored them for my particular theory. (Besides, I doubt that most NN people will want to adopt my terms because they like to think of me as the enemy.)
sarima@tdatirv.UUCP (Stanley Friesen) (10/13/90)
In article <A3FRYSG@cs.swarthmore.edu> gessel@cs.swarthmore.edu (Daniel Mark Gessel) writes: >You seem to be starting from the point of view that NNs embody the way the >human brain works. Unless there have been some huge jumps in the understanding >of the human brain and it's functioning, no one knows if NNs are even close. I was operating on the assumption that NN's were originally concieved as a sort of crude analog to the type of processing done by living things. I also assumed that the limitations of current NN's are due to limitations of technology and knowledge, not fundamental limitations of circuits. >How the exact chemical composition of the brain affects the functioning of >the brain is unknown. Todays artificial NN cannot be assumed to capture this >accurately. To assume that future NN's will be able to is to assume that we >will be able to recreate the human brain via well defined functions (in the >Mathematical sense) which is not necessarily possible. It is true there are many unknowns in neurobiology, but I do not see that there is any reason to believe that neurons do anything that is not at least amenable to aproximation using electronics. Indeed, I suspect that they do not even do anything that cannot be exactly duplicated. But even more important, I suspect that the exact details of biological neurobiology are irrelevant to making a general purpose responsive network [such as NN's]. That is, the redundancy and intrinsic variability in neuron response patterns make exact duplication unnecessary. In basic operation a neuron appears to be a complex combinatorial circuit. It performs some sorted of a generalized weighted 'sum' of an enormous number of inputs (often several thousand), and distributes the resulting signal to a large number destinations (again often in the thousands). The summation is probably quite different than the simple arithmetic sum used by current NN's, and the weighting and threshold functions are probably more complex than we currently imagine, but this is just a matter of technology and knowledge, not metaphysics. -- --------------- uunet!tdatirv!sarima (Stanley Friesen)
petkau@herald.UUCP (Jeff Petkau) (10/14/90)
From article <21128@well.sf.ca.us>, by nagle@well.sf.ca.us (John Nagle): > It's also worth bearing in mind that nothing like the backward- > propagation learning of the NN world has yet been discovered in biology. > The mechanism found so far look much more like collections of > adaptive controllers operating control loops. However, it should > be noted that most of the neural structures actually understood are > either in very simple animals (like slugs) or very close to sensors > and actuators (as in tactile control), where one would expect > structures that work like adaptive feedback control systems. > John Nagle Not entirely true. In "Memory Storage and Neural Systems" (Scientific American, July 1989) Daniel Alkon describes how rabbit neurons change in response to Pavlovian conditioning. The basic mechanism is: if neuron A fires and nearby neuron B happens to fire half a second later, a link will gradually form such that the firing of B is triggered by the firing of A, even in the abscence of whatever originally triggered B. Although this isn't quite the same as back-propogation, in simulated neural nets it actually seems to work far better (learning times are greatly reduced), and has the added advantage that knowledge of the final "goal" is not required. It also corresponds (in my mind at least) very closely to the observed behaviour of living things (mostly my cat). As a basic example of how such a net can be trained, I'll use a character recognizer. Start with a net with a grid of inputs for the pixel data and a second set of inputs for, say, the ASCII code of the characters (obviously ASCII isn't the best way to do it, but it keeps this post shorter). You also have a set of outputs for the network's guess. You start by hardwiring the network so that the ASCII inputs are wired directly to the ASCII outputs: input hex 4C and you'll see hex 4C on the output. Now, all you have to do is continually present pictures of characters at the pixel input along with their correct ASCII representations. Thus, when the network sees a capital L, it is forced to output a hex 4C. It soon learns to apply the outputs without benefit of the guiding input, and without the use of artificial devices like back propogation. [Sorry if this is old news in c.a.n-n, but it's getting a bit off topic for c.a.p]. Jeff Petkau: petkau@skdad.USask.ca Asterisks: ***********************
holt@auk.cs.athabascau.ca (Peter D Holt) (10/16/90)
sarima@tdatirv.UUCP (Stanley Friesen) writes: >I was operating on the assumption that NN's were originally concieved as >a sort of crude analog to the type of processing done by living things. >I also assumed that the limitations of current NN's are due to limitations >of technology and knowledge, not fundamental limitations of circuits. >It is true there are many unknowns in neurobiology, but I do not see that >there is any reason to believe that neurons do anything that is not at >least amenable to aproximation using electronics. Indeed, I suspect that >they do not even do anything that cannot be exactly duplicated. >But even more important, I suspect that the exact details of biological >neurobiology are irrelevant to making a general purpose responsive network >[such as NN's]. That is, the redundancy and intrinsic variability in neuron >response patterns make exact duplication unnecessary. In basic operation >a neuron appears to be a complex combinatorial circuit. It performs some >sorted of a generalized weighted 'sum' of an enormous number of inputs >(often several thousand), and distributes the resulting signal to a large >number destinations (again often in the thousands). The summation is >probably quite different than the simple arithmetic sum used by current NN's, >and the weighting and threshold functions are probably more complex than >we currently imagine, but this is just a matter of technology and knowledge, >not metaphysics. >-- >--------------- >uunet!tdatirv!sarima (Stanley Friesen) The above is a very acceptable opinion. However, I think previously you have mixed up your opinions with fact. Just because the above is your opinion, it does not provide scientific support the contention of your original posting that the human brain is an existence proof that neural nets can do anything people can do. Without defining neural net such a statement is vacuous. If you define neural nets as what the brain has of course you are correct. BUT we do not know how exactly how those function or more importantly how they support all of cognition so we do not know if we can simulate that support. If you define neural networks in anyway approximating what I have seen or read of current simulations (or their approximating logical extensions) I suspect that you are wrong. BUT I do not even have to prove that, by any current standards of science it is up to you to demonstrate that you are correct. Since you cannot do that this would seem to imply that if there are any "metaphysics" involved here it is that neural nets have become somewhat of a religion for you. I do not mean to imply that synthetic or simulated neural nets are not a very interesting technology, nor that they do not have endearing attributes not found not found in symbolic AI. Nor that they may successfully models some aspects of human perception and cognition, I just question the usefulness of making claims for neural nets that are unsubstantiable.
pollack@dendrite.cis.ohio-state.edu (Jordan B Pollack) (10/18/90)
Marvin Minsky has turned up the heat in this newsgroup, with the folksy "lets not choose sides" attack on connectionism he offered in PERCEPTRONS'88. Since a few people have already argued these points (or conceded to his authority!) I will collect most of the arguable material in one place: 1) He (again) chose non-recurrent backprop as a strawman representative of the entire field. He also claimed that 2) nothing useful could come out of the weights of a network, that 3) the field is "muddled" on terminology for "compound" architectures, and that 4) the central goal of the field is to find a homogeneous NN to solve the tabula-rasa-to-genius learning problem. I briefly respond: 1) the strawman can be ignored because he carefully hedged the overgeneralization gambit (this time) with parenthetical references to recurrent networks. 2) several researchers have found classical algorithms (principal components, hierarchal clustering) implemented in the procedures of networks.(e.g. Cottrell, Granger) 3) many researchers work on "architectures" composed of "modules" without getting muddled,(e.g. Jacobs, Ballard) 4) this is equivalent to accusing AI of having the goal of programming just one genius program. But most of this noise is still about the decades-old controversy over the relative promise of the bottom-up and top-down approaches to studying cognition. This promise can be assessed by seeing what sort of corner your theory backs you into: >>Where is the >>"traditional, symbolic, AI in the brain"? The answer seems to have >>escaped almost everyone on both sides of this great and spurious >>controversy! The 'traditional AI' lies in the genetic specifications >>of those functional interconnections: the bus layout of the relations >>between the low-level networks. A large, perhaps messy software is >>there before your eyes, hiding in the gross anatomy. I have to admit this is definitely a novel version of the homunculus fallacy: If we can't find him in the brain, he must be in the DNA! Of all the data and theories on cellular division and specialization and on the wiring of neural pathways I have come across, none have indicated that DNA is using means-ends analysis. Certainly, connectionist models are very easy to decimate when offered up as STRONG models of children learning language, of real brains, of spin glasses, quantum calculators, or whatever. That is why I view them as working systems which just illuminate the representation and search processes (and computational theories) which COULD arise in natural systems. There is plenty of evidence of convergence between representations found in the brain and backprop or similar procedures despite the lack of any strong hardware equivalence (Anderson, Linsker); constrain the mapping task correctly, and local optimization techniques will find quite similar solutions. Furthermore, the representations and processes discovered by connectionist models may have interesting scaling properties and can be given plausible adaptive accounts. On the other hand, I take it as a weakness of a theory of intelligence, mind or language if, when pressed to reveal its origin, shows me a homunculus, unbounded nativism, or some evolutionary accident with the same probability of occurrence as God. (Chomsky's corner). So, the age-old question of conflicting promise can be rephrased as follows, either being a valid philosophical approach to AI: Should we study search and representation as they occur in nature, or as algorithm and data-structure in artificial symbolic systems? I choose the former, since nature seems to have solved many problems which continue to haunt the not-so-young-anymore engineering field of AI. -- Jordan Pollack Assistant Professor CIS Dept/OSU Laboratory for AI Research 2036 Neil Ave Email: pollack@cis.ohio-state.edu Columbus, OH 43210 Fax/Phone: (614) 292-4890
danforth@riacs.edu (Douglas G. Danforth) (10/20/90)
In <JMC.90Oct8083235@Gang-of-Four.usenet> jmc@Gang-of-Four.usenet (John McCarthy) writes: >As >a branch of mathematics, logic is grounded in formal semantics as >Tarski and others have described, i.e. it has been made independent >of the thought processes that motivated its development. If Dr. McCarthy means that logic is now a thought process that is different from earlier less precise thought processes then I agree. If, however, he means that logic is now independent of all thought processes and that it somehow stands outside of "thought" and has an existence (in some Platonic sense) of its own then I disagree. If all intelligent life ceased to exist then what becomes of logic? Would it be the marks on pages of books fluttering in the wind of silent planets? Logic exists in the "social sea" of living creatures. -- Douglas G. Danforth (danforth@riacs.edu) Research Institute for Advanced Computer Science (RIACS) M/S 230-5, NASA Ames Research Center Moffett Field, CA 94035