camargo@cs.columbia.edu (Francisco Camargo) (10/19/89)
Hi there, I'm trying to find evidence for the existence in the brain of SIGMA-PI units as suggested in the PDP book. The real question is: "Are there neurons in the real brain that support the current research on higher order networks ? " I'm puzzled with the difficulty to implement simple nets capable of outputting a continuous quantity proportional to the product of its inputs. Even in the simpler form (multiplying X and Y to produce K.Z for some K in R), this problem has taken hours in a back-prop simulator, yielding very poor precision even with 'apparently' complex nets (3 to 5 layers, 3 to 10 neurons per layer). Any comments are appreciated. Francisco A. Camargo camargo@cs.columbia.edu
honavar@goat.cs.wisc.edu (A Buggy AI Program) (10/21/89)
In article <384@cs.columbia.edu> camargo@cs.columbia.edu (Francisco Camargo) writes: >Hi there, > >I'm trying to find evidence for the existence in the brain of SIGMA-PI units >as suggested in the PDP book. The real question is: > > "Are there neurons in the real brain that support the current research > on higher order networks ? " There is evidence for multiplicative interactions on the dendrites of neurons i.e., local regions of dendritic arbors can act like multiplicative units (not necessarily sigma-pi) whose outputs are summed at the cell body. For an excellent review of this and related issues, see the paper by Gordon Shepherd titled "The significance of real neuron architectures for neural network simulations" in "Computational Neuroscience" ed. E. Schwartz, 1989 (to appear). Vasant Honavar
mehra@s.cs.uiuc.edu (10/24/89)
? "Are there neurons in the real brain that support the current research ? on higher order networks ? " ? ?I'm puzzled with the difficulty to implement simple nets capable of outputting ?a continuous quantity proportional to the product of its inputs. Even in the ?simpler form (multiplying X and Y to produce K.Z for some K in R), this ?problem has taken hours in a back-prop simulator, yielding very poor Also important to note is the fact that when "real brain" operations occur, they are in frequency domain. Anyone with even a smattering of knowledge about signal processing (like me) knows that multiplication in time domain is convolution in frequency domain. So, what might seem like a complex operation (multiplication of numbers) might be realized rather easily in proper frequency domain circuitry... Now, of course, what with DSP and all, everyone seems to be using non-neural hardware for doing this stuff. Maybe someone from analog VLSI knows the answer. Of course this is speculative, and it is not saying that this is what Rumelhart et al. had in mind when they wrote their book. Pankaj Mehra
mehra@s.cs.uiuc.edu (11/11/89)
Recently, there was some discussion about higher order nets and the difficulty of realizing multiplication using neural mechanisms. One possibility I suggested was that of doing multiplication through convolution in frequency domain. Some people asked me for a reference to some paper showing the use of frequency domain operations in "real brains". Here is one: Neuroethology of Acoustic Prey Localization in the Barn Owl by Masakazu Konishi in Neuroethology and Behavioral Psychology eds F Huber & H. Markl (c) Springer-Verlag 1983 see p. 307