Neural Networks - Pointers to good texts?
Jordan Bortz
I'm looking for some good texts and/or articles on neural networks,
in English, and preferably focusing on real life algorithms/implementations
rather than obscure mathematics.
If you know of any, please let me know by mail, and I'll summarize
to the net; as this topic seems to be generating more interest.
I know some articles were mentioned earlier, but what about others?
Jordan
--
=============================================================================
Jordan Bortz Higher Level Software 1085 Warfield Ave Piedmont, CA 94611
(415) 268-8948 UUCP: (decvax|ucbvax|ihnp4)!decwrl!sun!plx!titn!jordan
=============================================================================
William Calvin
Best book on neural networks is THE CRUSTACEAN STOMATOGASTRIC GANGLION by
Selverston and Moulins (Springer 1987). If you mean neural-like networks,
try the Rumelhart et al PARALLEL DISTRIBUTED PROCESSING (MIT Press 1986).
We brain researchers sure get tired of hearing neural-like networks
referred to as "neural networks", an established subject for 25 years since
the days of Limulus lateral inhibition. Calling silicon networks "neural" is
about like the hype in the early days when every digital computer was
called a "brain" by the media.
William H. Calvin
University of Washington NJ-15, Seattle WA 98195
wca...@well.uucp wca...@uwalocke.bitnet
Frederick J Dickey
> We brain researchers sure get tired of hearing neural-like networks
> referred to as "neural networks", an established subject for 25 years since
> the days of Limulus lateral inhibition.
I think the above says that "biological" neural nets have been studied as a
formal discipline for 25 years and that this great ancestry gives biology
prior claim to the term "neural nets". Assuming that this is a correct
interpretation, let me make the following observation. In 1943, McCulloch
and Pitts published a paper entitled "A logical calculus of the ideas
immanent in neural nets". Minsky and Papert (Perceptrons) state that this
paper presents the "prototypes of the linear threshold functions". This paper
stikes me as clearly being in the "neural net-like" tradition. Now
1987-1943 = 44. Also note that 44 > 25. Therefore, it apears that the
"neural net-like" guys have prior claim to the term "neural net". :-).
David E. Leasure
wca...@well.UUCP (William Calvin) writes:
> We brain researchers sure get tired of hearing neural-like networks
>referred to as "neural networks", an established subject for 25 years since
>the days of Limulus lateral inhibition. Calling silicon networks "neural" is
>about like the hype in the early days when every digital computer was
>called a "brain" by the media.
Maybe we could all agree on a more faithful/less ingratiating term?
Maybe connectionist processing models or neuromorphic (after
Touretzky), or fine-grained parallel processing? (even Rumelhart's
Parallel Distributed Processing?)
David E. Leasure
Rutgers/AT&T
Stephen Smoliar
Well . . . this is all rather silly. The PUBLISHED title of the classic
paper by McCullogh and Pitts is "A Logigal Calculus of the Ideas Immanent
in Nervous Activity." They NEVER use "neural net" as a technical term
(or in any other capacity) in the paper. They ARE, however, concerned
with a net model based on the interconnection of elements which they call
neurons--appealing to properties of neurons which were known at the time
they wrote the paper. Personally, I think Calvin has a point. Investigators
who are searching the literature will probably benefit from cues which
distinguish papers about actual physiological properties from those about
computational models of those properties.
William Calvin
I thank you all for the suggestions regarding renaming non-neural "Neural
Networks" -- perhaps we can continue the discussion in the newsgroup
comp.ai.neural-nets rather than here in comp.ai as such.
gold...@osiris.cso.uiuc.edu
I agree with the respondent whose user-name was listed as "smoliar" that this
haggling about earliest references is "silly". In fact, I don't understand
the need for any territorial fight over terminology here.
Do physiologists actually use the two-word term "neural network" in their
literature? "Neuron", and "neural tissue", surely, but do they actually use
"neural network" ? If not, then there is no ambiguity. Sure there is some
danger of confusion, but no more than I think is usual in cases of "learned
borrowing". The term "neural network" as used by "connectionist/AI"
researchers caught on precisely because this model of computation is based on
the gross behavior of real, mammalian-brain neurons. It can be viewed in some
ways as a study of the human brain itself. Thus it is no greater an abuse of
terminology than, for example, "pipeline computers".
On the other hand, whatever became of the term "cybernetics" that Norbert
Weiner coined long ago? I thought its definition was quite suitable for
denoting this research. I doubt that "connectionist" is much help, in view of
the fact that the "connection machine" is more a project in pure parallelism
than intended as a neural model.
If I am wrong about any of this, please enlighten me.
Stephen Smoliar
>
> On the other hand, whatever became of the term "cybernetics" that Norbert
>Weiner coined long ago? I thought its definition was quite suitable for
>denoting this research.
I do not profess to be an expert in either the history of cybernetics or the
usage of the term; but, with that qualification, let me try to address this
question. As I recall, Weiner's original concern was with the design of
analog devices which, by virtue of feedback circuits, were capable of control
of other devices and adaptive behavior (which may be regarded as self-control).
Through my encounters with the literature as an AI researcher, I have observed
that the term "cybernetics" appears with greater frequency in Europe
(particularly the Soviet Union and the United Kingdom) than it does in
the United States. There is definitely a tendency to recognize that
Weiner's original principles could be generalized from analog to digital
hardware. However, I have the distinct impression that cybernetics grew
from the belief that behavioral knowledge was something which would ultimately
be encoded in the feedback loops, rather than in an explicit device concerned
with memory or the storage of a knowledge base. I would appreciate any
reactions to these comments simply to get the historical record straight.
Frederick J Dickey
From postnews Wed Oct 21 07:49:49 1987
> >interpretation, let me make the following observation. In 1943, McCulloch
> >and Pitts published a paper entitled "A logical calculus of the ideas
> >immanent in neural nets". Minsky and Papert (Perceptrons) state that this
>
> Well . . . this is all rather silly. The PUBLISHED title of the classic
> paper by McCullogh and Pitts is "A Logigal Calculus of the Ideas Immanent
> in Nervous Activity." They NEVER use "neural net" as a technical term
Well, this is very interesting. When I read Calvin's original
posting I was struck by the claim that neural nets had been studied for
25 years. This surely seemed too small a figure to me. To check this
out without initiating a major research project, I grabbed a copy of
Minsky and Papert's "Perceptrons" which happened to be on my desk at the
time and opened to the bibliography. M&P give the title of the McCullough and
Pitts paper as "A logical calculus of the ideas immanent in neural nets".
I'm looking at it right now and that's what it says. Apparently, the citation
is wrong. Well, I stand corrected.
I might comment by the way that regardless of the merits of Calvin's claim
that artificial neural nets ought to be named something else, I think the
effort is doomed to failure. The reason being that we seem to have become
an excessively marketing-oriented society. Labels are attached to things to
stimulate desired responses in "consumers," not to clarify crtical
distinctions. The practical problem one faces in much of the industrial world
is attempting to gain support for one's "research." To do this, one presents
one's proposed program to a manager, i.e., one markets it. The noise level
in this process is so high that nothing less than hype makes it through.
My experience with managers leads me to believe that they may have heard
of neural nets. If I tried to start a "neuroid" project, they would say
"Isn't that the same thing as a neural net?" I can guarantee you that they
aren't interested in the distinctions between artifical and biological nets.
How can an aerospace company make a profit from biological nets? In other
words, to start a artifical neural net project, I have to call it a neural
net, show how it applies to some product, how it adds value to the product
(neural nets will make the product more powerful than a locomotive, faster
than a speeding bullet, and able to leap over tall buildings at a single
bound), and how all this can be done by next year at a half man-year level
of effort.
If I lived in an ivory tower (a not unpleasant domicile), I'd say that
Calvin is right on. Out here in the cinder block towers, he's out to lunch.
To summarize, I'm sympathic to his viewpoint, but my sympathy isn't going
to make much difference.
A Buggy AI Program
>In article <830...@osiris.cso.uiuc.edu> gold...@osiris.cso.uiuc.edu writes:
>>
>> On the other hand, whatever became of the term "cybernetics" that Norbert
>>Weiner coined long ago? I thought its definition was quite suitable for
>>denoting this research.
>
>analog devices which, by virtue of feedback circuits, were capable of control
>of other devices and adaptive behavior (which may be regarded as self-control).
>Through my encounters with the literature as an AI researcher, I have observed
>that the term "cybernetics" appears with greater frequency in Europe
>(particularly the Soviet Union and the United Kingdom) than it does in
>the United States. There is definitely a tendency to recognize that
>Weiner's original principles could be generalized from analog to digital
>hardware. However, I have the distinct impression that cybernetics grew
>from the belief that behavioral knowledge was something which would ultimately
>be encoded in the feedback loops, rather than in an explicit device concerned
>with memory or the storage of a knowledge base. I would appreciate any
>reactions to these comments simply to get the historical record straight.
Moscow, 1986 - which by the way, is an eminently readable book):
"study of control and communication in machines and human beings" -
-- Norbert Weiner, USA.
"a science concerned with the study of systems of any nature which are
capable of receiving, storing, and processing information so as to use
it for control"
-- Academician A. N. Kolmogorov, USSR.
"the art of securing efficient operation"
-- L. Couffignal, France.
"a general theory of causality which is interpreted accurately to the
part of isomorphism"
-- A. Markov, Associate fellow, USSR Academy of Sciences.
"a science concerned with the control of sophisticated dynamic systems,
which is theoretically based on mathematics and logic, and practically,
on the use of means of automation, electronic computers of primarily
control and data-processing types"
-- Axel Berg
"a science concerned with the laws of receiving, storing, transmitting,
and processing information in sophisticated systems of control"
-- Academician V. Glushkov, USSR.
"a science concerned with systems that have vitality, that is, which
behave so as to survive"
-- Stafford Beer, British mathematician
-- Vasant Honavar
(hon...@speedy.cs.wisc.edu)