schiebel@a.cs.wvu.wvnet.edu (Darrell Schiebel) (02/06/90)
Could someone recomend a "good" book at a moderate level which provides a mathemetical framework for computational/"non-AI" learning, covering the implications from statistics/cybernetics/ thermodynamics/etc.? These and undoubtedly many other fields seem to provide the theoretical core for this field of research. (?) I would greatly appreciate any assistance. (BTW) Does anyone know the scope, depth, or quality of Nils Nilsson's "Mathemetical Foundations of Learning Machines" (Morgan Kaufmann) ??? Many Thanks, Darrell Schiebel (schiebel@a.cs.wvu.wvnet.edu) (UN027714@wvnvaxa.wvnet.edu) (drs@baks.com)
cjoslyn@bingvaxu.cc.binghamton.edu (Cliff Joslyn) (02/08/90)
In article <689@h.cs.wvu.wvnet.edu> schiebel@a.cs.wvu.wvnet.edu (Darrell Schiebel) writes: >I would greatly appreciate any assistance. (BTW) Does anyone know the >scope, depth, or quality of Nils Nilsson's "Mathemetical Foundations of >Learning Machines" (Morgan Kaufmann) ??? The one I know is just called /Learning Machines/ by Nilson, and it's out of print. It covers classic perceptrons, classifiers, committee machines, linear and quadratic, parametric training methods, layered machines, etc. It was OK, but I'm not into the stuff. -- O-------------------------------------------------------------------------> | Cliff Joslyn, Cybernetician at Large, cjoslyn@bingvaxu.cc.binghamton.edu | Systems Science, SUNY Binghamton, Box 1070, Binghamton NY 13901, USA V All the world is biscuit shaped. . .
heck@Sunburn.Stanford.EDU (Stefan P. Heck) (02/08/90)
AUTHOR: Nilsson, Nils J.
TITLE: Learning machines; foundations of trainable pattern-classifying
systems. By Nils J. Nilsson.
IMPRINT: McGraw-Hill, 1965.
137 p.
TOPICS: Artificial intelligence
NOTES: Item CSUU1729870-B (Books) Language: Year:
I hope this helps. It seems still available....
Stefan Heck
Symbolic Systems, Stanfordshankar@rnd.GBA.NYU.EDU (Shankar Bhattachary) (02/08/90)
The original posting asked about this in combination with classifier
systems, if I remember correctly.
In "Parallel Architectures and Neural Networks"
(proceedings of the first Italian workshop, April 27-29, 1988),
E.R. Caianiello, ed,
World Scientific (1989)
there is an article:
Classifier Systems and Neural Networks,
M. Compiani, D. Montanari, R. Serra & G. Valastro.
This article also refers to : Network Learning, by Fogelman, Gallinari,
Le Cun, and Thiria, to appear in :
Machine Learning, volume 3,
Kodratoff and Michalski, editors.
Does anyone know if this volume actually appeared? I have not been able to
track it down so far.
Also, could someone tell me where Michalski is ? I thought I knew, but I
was wrong.
Thanks.
--------------------------------------------------------------------------
Shankar Bhattacharyya, Information Systems, New York University
shankar@rnd.gba.nyu.edu
--------------------------------------------------------------------------bill@wayback.unm.edu (william horne) (02/09/90)
What come to be know as Computational Learning Theory is actually an offshoot of work by Valient. In fact, the Computational Learning Theory (COLT) Conferences, are centered around Valient's approach to learning. An introductory reference is: Valient, L.G., "A Theory of the Learnable", Comm. of the ACM, v27, n11 pp. 1134-1142. Nils Nilsson's book, Learning Machines, is kind of a precursor to conventional pattern recognition, for which a better book is clearly Duda & Hart, Pattern Classification and Scene Analysis
fulk@cs.rochester.edu (Mark Fulk) (02/09/90)
In article <1546@ariel.unm.edu> bill@wayback.unm.edu (william horne) writes: >What come to be know as Computational Learning Theory is actually an >offshoot of work by Valient. In fact, the Computational Learning Theory >(COLT) Conferences, are centered around Valient's approach to learning. >An introductory reference is: > >Valient, L.G., "A Theory of the Learnable", Comm. of the ACM, v27, n11 > pp. 1134-1142. > I am a person doing computational learning theory who work does NOT derive from Valiant's. In fact, I started in the field before he did. I am definitely a member of the COLT crowd, having published and presented at COLT, and being this year's conference chair. Computational learning theory includes any research into the mathematics of learning machines. Most research derives it inspiration from the work of E. Mark Gold and Raymond Solomonoff, in the late 60's. Gold's seminal paper is entitled "Language identification in the limit", and appeared in Information and Control in 1967. The field now includes the following strands: Recursion-theoretic (mainly what I do), which foreswears (for the time being, not permanently) considerations of efficiency. Our hope is to cast light on what might EVENTUALLY be a vigorous field of feasible learning theory. Our claim is that the recursion-theoretic setting, by deliberately ignoring issues such as run-time, permits easier examination of other issues which are at least as important. It seems to me that the most important issues involve the sorts of data used by a learning machine, and the criteria for its success. Concrete, (which I've done a bit of) which tries to find efficient mechanisms for learning specific classes of functions or sets. For example, how efficiently can one learn a regular language from positive and negative examples? Gold proved that finding the smallest DFA to fit some data is NP-complete, but I don't think that really answers the question (even assuming NP != P). Probabilistic concrete: Given some sort of random source of data, one is interested in algorithms that probably succeed, at least in supplying a good approximation. For example, learn a convex set in the plane from examples chosen randomly from the set according to the uniform distribution on the plane restricted to the set. (Easy: draw a bunch of examples, form the convex hull). This kind of learning is often called "probably approximately correct" (pac) learning. Valiant-style, or "distribution-independent" pac learning: One must give an algorithm that, given ANY random source of data about a problem it is supposed to work on, probably produces an approximately correct answer, where the approximation is measured with the same distribution used to produce the data. There are also questions of learning things other than programs, for example first-order formulae, and of very powerful kinds of data (f.o. formulae in restricted languages are quite popular again). Obviously I'm raving on about a favorite subject. For more information, consult the proceedings of the First and Second Workshop on Computational Learning Theory, published by Morgan-Kaufman. The literature appears mainly in Information and Computation (formerly Information and Control), with some articles in Journal of the ACM and Theoretical Computer Science, as well as Journal of Computer and System Sciences. The East German journal EIK (Elektronische Informatik und Kybernetik) is a good source for some of the work of people in Eastern Europe. For a REAL introduction to the subject, including a chance to talk to real originators like Valiant, Manuel Blum, John Case, and Rusins Freivalds,... COME TO ROCHESTER AUGUST 6-8, 1990!! COLT '90, the Third Workshop on Computational Learning Theory, will be held in Stouffer's Rochester Plaza Hotel, Rochester, New York, on those dates. Reception the evening of Aug. 5 (Sunday). Send me your address, and I'll make sure you get registration materials (email fine, I'll email you a form). Expect the forms towards the end of May or early June. EVEN BETTER: SUBMIT A PAPER. I'll post the call for papers after this article. We are very interested in papers that carry out mathematical analyses of neural net learning. We are even more interested in papers that expose fresh insights into the nature of or criteria for learning. If you think the old stuff is foolish, and you have the right idea to replace it, and you can write a good mathematical paper, SUBMIT IT! We (the program committee, I'm just a member) might even agree!! Just, please, do not subject us to pure complaints with no constructive suggestions, and do not send papers of the "I tried it twice, and it worked both times (well sort of the first time, but the second time REALLY)" sort. Oh, and don't send anything to me. John Case is the program committee chair, and has things set up to receive papers at the University of Delaware. Send them there; for addresses, read the call for papers. Incidentally, the Special Interest Groups in Automata and Computability Theory (SIGACT), and in Artificial Intelligence (SIGART), of the Association for Computing Machinery (ACM) are sponsors of COLT '90. Mark Fulk fulk@cs.rochester.edu ...!uunet!rochester!fulk Computer Science Department University of Rochester Rochester, NY 14627
fulk@cs.rochester.edu (Mark Fulk) (02/09/90)
CALL FOR PAPERS COLT '90 Third Workshop on Computational Learning Theory Rochester, NY August 6 - August 8, 1990 The third workshop on Computational Learning Theory will be held in Rochester, NY. The conference will be jointly sponsored by SIGACT and SIGART, and is expected to be similar in style to the previous such workshops held at MIT and UC/Santa Cruz. Registration at COLT '90 is open. It is expected that most papers will present rigorous formal analyses of theoretical issues in Machine Learning. Possible topics include, but are not limited to: resource and robustness analysis of learning algorithms, general learnability and non-learnability results in new and existing computational learning models, theoretical comparisons among learning models, and papers that connect learning theory with work in robotics, neural nets, pattern recognition and cryptography. R. Freivalds (Latvian State University, Riga) has agreed to present an invited talk; the program committee may consider more such. Authors should submit an extended abstract that consists of: A) cover page with title, authors' names, (postal and e-mail) addresses, and a 200 word summary. B) body not longer than 10 pages in twelve-point font. Be sure to include a clear definition of the model used, an overview of the results, and some discussion of their significance, including comparison to other work. Proofs or proof sketches should be included in the technical section. Authors should send 10 copies of their abstract to John Case COLT '90 Department of Computer and Information Sciences 103 Smith Hall University of Delaware Newark, DE 19716. The deadline for receiving submissions is April 9, 1990. This deadline is FIRM. Authors will be notified by May 22, final camera-ready papers will be due June 18, and this deadline is ABSOLUTE. The proceedings will be published by Morgan-Kaufmann. For further information about submissions contact John Case (telephone: 302-451-2711, email: case@cis.udel.edu). Chair and local arrangements: Mark A. Fulk (U. Rochester). Program committee: J. Case (Delaware, chair), D. Angluin (Yale), E. Baum (NEC Research, Princeton) S. Ben David (Technion, Israel), M. Fulk (U. Rochester), D. Haussler (UC Santa Cruz), L. Pitt (U. Illinois), R. Rivest (MIT), C. Smith (Maryland), S. Weinstein (U. Pennsylvania). Note: papers that have appeared in journals or that are being submitted to other conferences are not appropriate for submission to COLT with the exception of papers submitted to the IEEE 30th Symposium on Foundations of Computer Science (FOCS). A joint submission policy coordinated with FOCS permits authors to send a paper to both conferences; in the event that both conferences accept the paper, it will be published in the FOCS proceedings, the authors will be invited to give a talk at both conferences, and a short (one-page) abstract will be printed in the COLT proceedings. As the FOCS decisions may be quite late, authors of dual submissions will be asked to send the abstract with their final copy, so as to allow the publisher to substitute the abstract upon receiving word of the FOCS decision. It is, of course, required that authors notify both committees of the dual submission by including a note in the cover letters.
honavar@goat.cs.wisc.edu (Vasant Honavar) (02/11/90)
In article <2937@bingvaxu.cc.binghamton.edu> cjoslyn@bingvaxu.cc.binghamton.edu.cc.binghamton.edu (Cliff Joslyn) writes: > >The one I know is just called /Learning Machines/ by Nilson, and it's >out of print. It covers classic perceptrons, classifiers, committee >machines, linear and quadratic, parametric training methods, layered >machines, etc. It was OK, but I'm not into the stuff. "Learning Machines" is being reprinted by Morgan Kaufmann (available early 1990).