One reason to dismiss classical (say computational/symbolic/etc.) AI models of cognition is the fact that they cannot incorporate adaptive processes as evolution, development, learning (all this is possible with biologically motivated models like neural networks).
Does anybody know counterexamples, i. e. classical models that DO incorporate adaption?
> One reason to dismiss classical (say computational/symbolic/etc.) > AI models of cognition is the fact that they cannot incorporate > adaptive processes as evolution, development, learning > (all this is possible with biologically motivated models > like neural networks).
> Does anybody know counterexamples, i. e. classical models that > DO incorporate adaption?
Do you mean pure procedural AI? Where can such a beast still be found? It is as mythical as a Wolpertinger.
Adaptive pattern recognition and GA/NN learning are hot topics of connectionist AI. This stuff is/going to be classic already.
Pure inference kind of symbolic logic can't learn very well, ok. But have a look at Cyc. Deriving new rules from experience _is_ learning.
> Any defense of AI appreciated > Marc-Denis Weitze.
In article <3hnjto$...@sunserver.lrz-muenchen.de>, Marc D. Weitze <t383...@hp23.lrz-muenchen.de> wrote:
>One reason to dismiss classical (say computational/symbolic/etc.) >AI models of cognition is the fact that they cannot incorporate >adaptive processes as evolution, development, learning >(all this is possible with biologically motivated models >like neural networks).
Are you trying to claim that "biologically motivated models like neural networks" are not "computational/symbolic/etc."? If I tell you my "symbolic" learning system is biologically motivated, will you grant me that it can adapt and learn? There are a number of "classical" "symbolic" learning systems (presumably not "biologically motivated") that adapt their behavior with experience. For a number of examples, see:
Shavlik, J. W., \& Dietterich, T. G. (Eds.). (1990). Readings in machine learning. San Mateo, CA: Morgan Kaufmann.
What properties of "biologically motivated" models do you believe give them this otherwise unimplementable ability to adapt?
> One reason to dismiss classical (say computational/symbolic/etc.) > AI models of cognition is the fact that they cannot incorporate > adaptive processes as evolution, development, learning > (all this is possible with biologically motivated models > like neural networks).
First a few questions regarding the words you use. What do you mean by a computational model -- In what sense are neural networks (especially artificial NN) NOT computational?? What do you mean with learning in this context? What can an artificial neural net do, that can't be done by a symbolic machine learning system?
Indeed in the comparisons I know of, even (simple) machine learning algorithms like ID3 did clearly outperform Neural Nets (e.g. back- propagation nets) on the same task. This is not only the case wrt efficiency of the learning (which is rather trivial ;-) ), but also with respect to the quality of the results of learning. (This has been the case with the learning in the context of numerical data; a situation primed in favor of neural nets.)
Besides, have you ever tried to make a theorem prover (or any other symbolic system) learn using neural nets?
In article <1995Feb16.194...@informatik.uni-kl.de> sch...@informatik.uni-kl.de (Klaus Schmid) writes: > One reason to dismiss classical (say computational/symbolic/etc.) > AI models of cognition is the fact that they cannot incorporate > adaptive processes as evolution, development, learning > (all this is possible with biologically motivated models > like neural networks).
KS> First a few questions regarding the words you use. What do you KS> mean by a computational model -- In what sense are neural networks KS> (especially artificial NN) NOT computational?? What do you mean KS> with learning in this context? What can an artificial neural net KS> do, that can't be done by a symbolic machine learning system?
KS> Indeed in the comparisons I know of, even (simple) machine KS> learning algorithms like ID3 did clearly outperform Neural Nets KS> (e.g. back- propagation nets) on the same task.
Could you post a citation in support of this claim, please?
-- Dave -- Yeats on USEnet: Things fall apart; the centre cannot hold; Mere anarchy is loosed upon the world, The blood-dimmed tide is loosed and everywhere the ceremony of innocence is drowned; The best lack all conviction, while the worst are full of passionate intensity.
sch...@informatik.uni-kl.de (Klaus Schmid) writes: >> One reason to dismiss classical (say computational/symbolic/etc.) >> AI models of cognition is the fact that they cannot incorporate >> adaptive processes as evolution, development, learning >> (all this is possible with biologically motivated models >> like neural networks). >First a few questions regarding the words you use. What do you mean by >a computational model -- In what sense are neural networks (especially >artificial NN) NOT computational?? >What do you mean with learning in this context? What can an artificial >neural net do, that can't be done by a symbolic machine learning system?
Of course, biologically motivated models like ANNs are computational in the sense that they run on digital computers. But this is peripheral, for computers are only a tool in ANN modeling. The point is that ANNs can "show how a system might convert a meaningful input into a meaningful output without any rules, principles, inferences, or other sorts of meaningful phenomena in between." (Searle, 'Rediscovery' (1992), p. 246) This stands in contrast to computational models.
Biologically motivated ANNs lend themselves to adaptation, because adaptation is a biological category: Learning (changing synapses), development, and evolution (GAs) is easily transferable from biology to ANNs. This is not the case for classical GOFAI architectures: Adaptation does not play an 'intrinsic' role in computation/logic/language/etc. and cannot be incorporated in those models free and easy.
>Indeed in the comparisons I know of, even (simple) machine learning >algorithms like ID3 did clearly outperform Neural Nets (e.g. back- >propagation nets) on the same task. >This is not only the case wrt efficiency of the learning (which >is rather trivial ;-) ), but also with respect to the quality of >the results of learning. (This has been the case with the learning >in the context of numerical data; a situation primed in favor of >neural nets.)
Bobrow and Winograd found a nice metaphor in 1977: "Current systems, even the best ones, often resemble a house of cards... The result is an extremly fragile structure which may reach impressive heights, but collapses immediately if swayed in the slightest from the specific domain (often even the specific example) for which it was built (Cognitive Science, Vol.1, p. 4).
It is worth further examination if it be correct to label algorithmic processes (e.g. in ID3) as 'learning'.
> One reason to dismiss classical (say computational/symbolic/etc.) > AI models of cognition is the fact that they cannot incorporate > adaptive processes as evolution, development, learning > (all this is possible with biologically motivated models > like neural networks).
> Does anybody know counterexamples, i. e. classical models that > DO incorporate adaption?
Surely Case Based Reasoning is a case of a classical (ish) AI model that attempts to provide adaption.
@ARTICLE{CBR:intro:Kolodner, AUTHOR = "Janet L. Kolodner", TITLE = "An Introduction to Case Based Reasoning", JOURNAL = "Artificial Intelligence Review", YEAR = "1992", VOLUME = "6", NUMBER = "", PAGES = "3 - 34", NOTES = "A general intro article.", QUOTES ="", REFERS = ""}
@book{CBR:book:kolodner, author = "Janet L. Kolodner", year = 1993, publisher = "Morgan Kaufmann", isbn = "1 55860 237 2", title = "{Case-Based Reasoning}", notes =""}
@BOOK{inside:CBR, AUTHOR = " Christopher K. Reisbeck and Roger C. Schank", TITLE = "Inside Case-Based Reasoning", YEAR = "1989", EDITION = "", PAGES = "", PUBLISHER = "Lawrence Erlbaum", NOTES = "", QUOTES =" The ultimate task of a case based reasoner is to adapt the solution stored in a retrieved case to the needs of the current input. - p41 ", REFERS = ""}
In article <DBP.95Feb18082...@proof.csli.stanford.edu>, d...@csli.stanford.edu (David Barker-Plummer) writes: |> In article <1995Feb16.194...@informatik.uni-kl.de> sch...@informatik.uni-kl.de (Klaus Schmid) writes: |> |> > One reason to dismiss classical (say computational/symbolic/etc.) |> > AI models of cognition is the fact that they cannot incorporate |> > adaptive processes as evolution, development, learning |> > (all this is possible with biologically motivated models |> > like neural networks). |> |> KS> First a few questions regarding the words you use. What do you |> KS> mean by a computational model -- In what sense are neural networks |> KS> (especially artificial NN) NOT computational?? What do you mean |> KS> with learning in this context? What can an artificial neural net |> KS> do, that can't be done by a symbolic machine learning system? |> |> KS> Indeed in the comparisons I know of, even (simple) machine |> KS> learning algorithms like ID3 did clearly outperform Neural Nets |> KS> (e.g. back- propagation nets) on the same task. |> |> Could you post a citation in support of this claim, please? |> For, what is perhaps the most stricking evidence, I do not have a citation: In an informal meeting here at the University of Kaiserslautern some people reported about a project at their University (I think it was Dortmund). They implemented about a dozen machine learning and used them under identical conditions on the same task (prediction of the direction stock market prices go). On this task they used a back-propagation net, ID3, ID3 with pruning, etc. (a total of about a dozen algorithms - I do not recall them all at the moment). Among these back-propagation was among the (5? -I'm recalling this from memory) worse algorithms rated by prediction quality. However, along a different dimension it was leading: It needed about 100times as much cpu-time for learning than the second-worst algorithm. (And this, also the neural networks are usually regarded as a good candidate for this kind of task.)
BUT, if you want a citation, here we go: Have a look at
Gregory Piatetsky-Shapiro and William Frawley (eds.) Knowledge Discovery in Databases AAAI Press/The MIT Press, 1991
This is a collection of articles. All authors use "classical" machine learning algorithms (depending on what you define to be a "classical" algorithm). However, some of them also used connectionist approaches on the same task, but they do not report any improvement.
At the moment I do not have the book at hand, and at the time I was going through the articles I was not interested that much in NN, therefore I do not have the exact references (which articles did take NN into the comparison).
In article <3ia1g5$...@sunserver.lrz-muenchen.de>, t383...@hp22.lrz-muenchen.de (Marc D. Weitze) writes: |> Of course, biologically motivated models like ANNs are computational |> in the sense that they run on digital computers. But this is peripheral, |> for computers are only a tool in ANN modeling. The point is that ANNs |> can "show how a system might convert a meaningful input into a meaningful |> output without any rules, principles, inferences, or other sorts of |> meaningful phenomena in between." (Searle, 'Rediscovery' (1992), p. 246) |> This stands in contrast to computational models. There are rules, principles, inferences in NN - or how would you call the learning rules the (programmed) ANN adheres to: Obviously they are rules in the same sense in which there exist rules (e.g.) in ID3 for selecting an attribute to test on. Besides this (so-called) classical learning algorithms are not subject to rules either. Their behaviour is dominated by the examples (in the sense this is true for NN ;-)
|> Biologically motivated ANNs lend themselves to adaptation, because |> adaptation is a biological category: Learning (changing synapses), development, |> and evolution (GAs) is easily transferable from biology to ANNs. This |> is not the case for classical GOFAI architectures: Adaptation does not |> play an 'intrinsic' role in computation/logic/language/etc. and cannot |> be incorporated in those models free and easy. - What is a GOFAI architecture ? Here you are right - from a psychological point of view. I do think that the important criterion is, what performance the different architectures show at a learning task.
|> Bobrow and Winograd found a nice metaphor in 1977: |> "Current systems, even the best ones, often resemble a house of cards... |> The result is an extremly fragile structure which may reach impressive |> heights, but collapses immediately if swayed in the slightest from |> the specific domain (often even the specific example) for which it |> was built (Cognitive Science, Vol.1, p. 4). Indeed it is a NICE little metaphor, however, it goes besides the point. In 1977 none of the (so-called) classical learning algorithms for learning from examples existed. So obviously, they could not address this metaphor to these systems. Additionally, these algorithms have been used with success on a wide range of tasks, so what is said by this metaphor is obviously not correct for these systems. I have the impression that this metaphor was originally addressed to the expert systems of this time. For these it was true - as far as I know.
|> It is worth further examination if it be correct to label algorithmic |> processes (e.g. in ID3) as 'learning'. My point of view is that one can measure learning only by the I/O-behaviour of the system. Then because the (so-called) classical algorithms produce similar behaviour to this of ANN, consequently they learn, too.
However, if you like to decide this point based on the implementation. And decide that anything algorithmic is not learning, then you have to consider that ANN are no implementation of NN because they are algorithmic and NN are not algorithmic (because they learn and anything learning can not be algorithmic.)
