AI can't incorporate adaption
Marc D. Weitze
AI models of cognition is the fact that they cannot incorporate
adaptive processes as evolution, development, learning
(all this is possible with biologically motivated models
like neural networks).
Does anybody know counterexamples, i. e. classical models that
DO incorporate adaption?
Any defense of AI appreciated
Marc-Denis Weitze.
Eugen Leitl
On 13 Feb 1995, Marc D. Weitze wrote:
> One reason to dismiss classical (say computational/symbolic/etc.)
> AI models of cognition is the fact that they cannot incorporate
> adaptive processes as evolution, development, learning
> (all this is possible with biologically motivated models
> like neural networks).
>
> Does anybody know counterexamples, i. e. classical models that
> DO incorporate adaption?
Do you mean pure procedural AI? Where can such a
beast still be found?
It is as mythical as a Wolpertinger.
Adaptive pattern recognition and GA/NN learning
are hot topics of connectionist AI. This stuff
is/going to be classic already.
Pure inference kind of symbolic logic can't
learn very well, ok. But have a look at Cyc.
Deriving new rules from experience _is_
learning.
> Any defense of AI appreciated
> Marc-Denis Weitze.
>
CU,
Eugene Leitl.
Randolph M. Jones
Marc D. Weitze <t38...@hp23.lrz-muenchen.de> wrote:
>One reason to dismiss classical (say computational/symbolic/etc.)
>AI models of cognition is the fact that they cannot incorporate
>adaptive processes as evolution, development, learning
>(all this is possible with biologically motivated models
>like neural networks).
Are you trying to claim that "biologically motivated models like neural
networks" are not "computational/symbolic/etc."? If I tell you my
"symbolic" learning system is biologically motivated, will you grant me
that it can adapt and learn? There are a number of "classical"
"symbolic" learning systems (presumably not "biologically motivated") that
adapt their behavior with experience. For a number of examples, see:
Shavlik, J. W., \& Dietterich, T. G. (Eds.). (1990). Readings in machine
learning. San Mateo, CA: Morgan Kaufmann.
What properties of "biologically motivated" models do you believe give
them this otherwise unimplementable ability to adapt?
Follow-ups to comp.ai.philosophy
Klaus Schmid
> One reason to dismiss classical (say computational/symbolic/etc.)
> AI models of cognition is the fact that they cannot incorporate
> adaptive processes as evolution, development, learning
> (all this is possible with biologically motivated models
> like neural networks).
First a few questions regarding the words you use. What do you mean by
a computational model -- In what sense are neural networks (especially
artificial NN) NOT computational??
What do you mean with learning in this context? What can an artificial
neural net do, that can't be done by a symbolic machine learning system?
Indeed in the comparisons I know of, even (simple) machine learning
algorithms like ID3 did clearly outperform Neural Nets (e.g. back-
propagation nets) on the same task.
This is not only the case wrt efficiency of the learning (which
is rather trivial ;-) ), but also with respect to the quality of
the results of learning. (This has been the case with the learning
in the context of numerical data; a situation primed in favor of
neural nets.)
Besides, have you ever tried to make a theorem prover (or any other
symbolic system) learn using neural nets?
CU
Klaus
Simon Thompson
>
>Does anybody know counterexamples, i. e. classical models that
>DO incorporate adaption?
Yes,
ID3, Version spaces, Genetic programming and any other form of inductive,
deductive or abductive symbolic manipulation.
For instance
Si.
>
>Any defense of AI appreciated
>Marc-Denis Weitze.
AI does not need defending.
David Barker-Plummer
> One reason to dismiss classical (say computational/symbolic/etc.)
> AI models of cognition is the fact that they cannot incorporate
> adaptive processes as evolution, development, learning
> (all this is possible with biologically motivated models
> like neural networks).
KS> First a few questions regarding the words you use. What do you
KS> mean by a computational model -- In what sense are neural networks
KS> (especially artificial NN) NOT computational?? What do you mean
KS> with learning in this context? What can an artificial neural net
KS> do, that can't be done by a symbolic machine learning system?
KS> Indeed in the comparisons I know of, even (simple) machine
KS> learning algorithms like ID3 did clearly outperform Neural Nets
KS> (e.g. back- propagation nets) on the same task.
Could you post a citation in support of this claim, please?
-- Dave
--
Yeats on USEnet:
Things fall apart; the centre cannot hold;
Mere anarchy is loosed upon the world,
The blood-dimmed tide is loosed and everywhere
the ceremony of innocence is drowned;
The best lack all conviction, while the worst
are full of passionate intensity.
Marc D. Weitze
>> One reason to dismiss classical (say computational/symbolic/etc.)
>> AI models of cognition is the fact that they cannot incorporate
>> adaptive processes as evolution, development, learning
>> (all this is possible with biologically motivated models
>> like neural networks).
>First a few questions regarding the words you use. What do you mean by
>a computational model -- In what sense are neural networks (especially
>artificial NN) NOT computational??
>What do you mean with learning in this context? What can an artificial
>neural net do, that can't be done by a symbolic machine learning system?
Of course, biologically motivated models like ANNs are computational
in the sense that they run on digital computers. But this is peripheral,
for computers are only a tool in ANN modeling. The point is that ANNs
can "show how a system might convert a meaningful input into a meaningful
output without any rules, principles, inferences, or other sorts of
meaningful phenomena in between." (Searle, 'Rediscovery' (1992), p. 246)
This stands in contrast to computational models.
Biologically motivated ANNs lend themselves to adaptation, because
adaptation is a biological category: Learning (changing synapses), development,
and evolution (GAs) is easily transferable from biology to ANNs. This
is not the case for classical GOFAI architectures: Adaptation does not
play an 'intrinsic' role in computation/logic/language/etc. and cannot
be incorporated in those models free and easy.
>Indeed in the comparisons I know of, even (simple) machine learning
>algorithms like ID3 did clearly outperform Neural Nets (e.g. back-
>propagation nets) on the same task.
>This is not only the case wrt efficiency of the learning (which
>is rather trivial ;-) ), but also with respect to the quality of
>the results of learning. (This has been the case with the learning
>in the context of numerical data; a situation primed in favor of
>neural nets.)
Bobrow and Winograd found a nice metaphor in 1977:
"Current systems, even the best ones, often resemble a house of cards...
The result is an extremly fragile structure which may reach impressive
heights, but collapses immediately if swayed in the slightest from
the specific domain (often even the specific example) for which it
was built (Cognitive Science, Vol.1, p. 4).
It is worth further examination if it be correct to label algorithmic
processes (e.g. in ID3) as 'learning'.
>CU
>Klaus
Best
Marc-Denis Weitze.
andyd
On 13 Feb 1995, Marc D. Weitze wrote:
> One reason to dismiss classical (say computational/symbolic/etc.)
> AI models of cognition is the fact that they cannot incorporate
> adaptive processes as evolution, development, learning
> (all this is possible with biologically motivated models
> like neural networks).
>
> Does anybody know counterexamples, i. e. classical models that
> DO incorporate adaption?
Surely Case Based Reasoning is a case of a classical (ish) AI model
that attempts to provide adaption.
Have a look at:
@BOOK{CHEF,
AUTHOR = "Kristian Hammond",
TITLE = "Case Based Planning",
YEAR = "1989",
EDITION = "",
PAGES = "",
PUBLISHER = "Academic Press",
NOTES = "",
QUOTES ="",
REFERS = ""}
@ARTICLE{CBR:intro:Kolodner,
AUTHOR = "Janet L. Kolodner",
TITLE = "An Introduction to Case Based Reasoning",
JOURNAL = "Artificial Intelligence Review",
YEAR = "1992",
VOLUME = "6",
NUMBER = "",
PAGES = "3 - 34",
NOTES = "A general intro article.",
QUOTES ="",
REFERS = ""}
@book{CBR:book:kolodner,
author = "Janet L. Kolodner",
year = 1993,
publisher = "Morgan Kaufmann",
isbn = "1 55860 237 2",
title = "{Case-Based Reasoning}",
notes =""}
@BOOK{inside:CBR,
AUTHOR = " Christopher K. Reisbeck and Roger C. Schank",
TITLE = "Inside Case-Based Reasoning",
YEAR = "1989",
EDITION = "",
PAGES = "",
PUBLISHER = "Lawrence Erlbaum",
NOTES = "",
QUOTES ="
The ultimate task of a case based reasoner is to adapt the solution
stored
in a retrieved case to the needs of the current input. - p41
",
REFERS = ""}
Klaus Schmid
|> In article <1995Feb1...@informatik.uni-kl.de> sch...@informatik.uni-kl.de (Klaus Schmid) writes:
|>
|> > One reason to dismiss classical (say computational/symbolic/etc.)
|> > AI models of cognition is the fact that they cannot incorporate
|> > adaptive processes as evolution, development, learning
|> > (all this is possible with biologically motivated models
|> > like neural networks).
|>
|> KS> First a few questions regarding the words you use. What do you
|> KS> mean by a computational model -- In what sense are neural networks
|> KS> (especially artificial NN) NOT computational?? What do you mean
|> KS> with learning in this context? What can an artificial neural net
|> KS> do, that can't be done by a symbolic machine learning system?
|>
|> KS> Indeed in the comparisons I know of, even (simple) machine
|> KS> learning algorithms like ID3 did clearly outperform Neural Nets
|> KS> (e.g. back- propagation nets) on the same task.
|>
|> Could you post a citation in support of this claim, please?
|>
In an informal meeting here at the University of Kaiserslautern some people
reported about a project at their University (I think it was Dortmund).
They implemented about a dozen machine learning and used them under identical
conditions on the same task (prediction of the direction stock market prices go).
On this task they used a back-propagation net, ID3, ID3 with pruning, etc. (a total
of about a dozen algorithms - I do not recall them all at the moment). Among these
back-propagation was among the (5? -I'm recalling this from memory)
worse algorithms rated by prediction quality.
However, along a different dimension it was leading: It needed about 100times as
much cpu-time for learning than the second-worst algorithm.
(And this, also the neural networks are usually regarded as a good candidate for
this kind of task.)
BUT, if you want a citation, here we go: Have a look at
Gregory Piatetsky-Shapiro and William Frawley (eds.)
Knowledge Discovery in Databases
AAAI Press/The MIT Press, 1991
This is a collection of articles. All authors use "classical" machine learning
algorithms (depending on what you define to be a "classical" algorithm).
However, some of them also used connectionist approaches on the same task, but they
do not report any improvement.
At the moment I do not have the book at hand, and at the time I was going through
the articles I was not interested that much in NN, therefore I do not have the exact
references (which articles did take NN into the comparison).
Bye
Klaus
Klaus Schmid
|> Of course, biologically motivated models like ANNs are computational
|> in the sense that they run on digital computers. But this is peripheral,
|> for computers are only a tool in ANN modeling. The point is that ANNs
|> can "show how a system might convert a meaningful input into a meaningful
|> output without any rules, principles, inferences, or other sorts of
|> meaningful phenomena in between." (Searle, 'Rediscovery' (1992), p. 246)
|> This stands in contrast to computational models.
learning rules the (programmed) ANN adheres to: Obviously they are rules in the
same sense in which there exist rules (e.g.) in ID3 for selecting an attribute
to test on.
Besides this (so-called) classical learning algorithms are not subject
to rules either. Their behaviour is dominated by the examples (in the sense
this is true for NN ;-)
|> Biologically motivated ANNs lend themselves to adaptation, because
|> adaptation is a biological category: Learning (changing synapses), development,
|> and evolution (GAs) is easily transferable from biology to ANNs. This
|> is not the case for classical GOFAI architectures: Adaptation does not
|> play an 'intrinsic' role in computation/logic/language/etc. and cannot
|> be incorporated in those models free and easy.
- What is a GOFAI architecture ?
Here you are right - from a psychological point of view. I do think that the
important criterion is, what performance the different architectures show at a
learning task.
|> Bobrow and Winograd found a nice metaphor in 1977:
|> "Current systems, even the best ones, often resemble a house of cards...
|> The result is an extremly fragile structure which may reach impressive
|> heights, but collapses immediately if swayed in the slightest from
|> the specific domain (often even the specific example) for which it
|> was built (Cognitive Science, Vol.1, p. 4).
Indeed it is a NICE little metaphor, however, it goes besides the point.
In 1977 none of the (so-called) classical learning algorithms for learning
from examples existed. So obviously, they could not address this metaphor to
these systems.
Additionally, these algorithms have been used with success on a wide range of
tasks, so what is said by this metaphor is obviously not correct for these
systems.
I have the impression that this metaphor was originally addressed to the expert
systems of this time. For these it was true - as far as I know.
|> It is worth further examination if it be correct to label algorithmic
|> processes (e.g. in ID3) as 'learning'.
My point of view is that one can measure learning only by the I/O-behaviour
of the system.
Then because the (so-called) classical algorithms produce similar behaviour
to this of ANN, consequently they learn, too.
However, if you like to decide this point based on the implementation. And
decide that anything algorithmic is not learning, then you have to consider
that
ANN are no implementation of NN because they are algorithmic and NN are
not algorithmic (because they learn and anything learning can not be
algorithmic.)
|> Best
|> Marc-Denis Weitze.
Regards
Klaus