AI100 Essay Contest
236 views
Skip to first unread message
Peter Stone
Jan 29, 2023, 4:39:45 PM1/29/23
to cv...@lists.auth.gr, robocup-...@lists.robocup.org, ml-...@googlegroups.com, rl-...@googlegroups.com, learnin...@det.ua.pt, connect...@mailman.srv.cs.cmu.edu, ai-ro...@googlegroups.com, u...@engr.orst.edu
AI100 Prize: Early Career Essay Competition
The One Hundred Year Study on Artificial Intelligence (AI100) is a
longitudinal study of progress in AI and its impacts on society. A key
feature of the 2021 AI100 report was its commentary on what had changed
since the first report published in 2016.
As a way of laying the groundwork for the next report, planned for 2026,
the AI100 Standing Committee invites original essay submissions that
react directly to one or both of the AI100 reports. Essay application is
now open and will close on March 31, 2023.
Apply here: https://ai100.stanford.edu/prize-competition
___
Professor Peter Stone
Truchard Foundation Chair in Computer Science
University Distinguished Teaching Professor
Director, Texas Robotics
Associate Chair, Department of Computer Science office: 512-471-9796
The University of Texas at Austin mobile: 512-810-3373
2317 Speedway, Stop D9500 pst...@cs.utexas.edu
Austin, Texas 78712-1757 USA http://www.cs.utexas.edu/~pstone
Schmidhuber Juergen
Feb 9, 2023, 8:56:39 AM2/9/23
to ml-...@googlegroups.com
Thanks, Peter, for this announcement!
I read the "Appendix I: A Short History of AI (Annotated)” of the AI100 report. It starts with a disclaimer: "A complete and fully balanced history of the field is beyond the scope of this document.”
In fact, it almost completely fails to credit the true pioneers of practical AI (including deep neural nets and computational hardware) and AI theory. Let me try to correct some of the most misleading statements. All references below can be found in the 2022 survey:
Annotated History of Modern AI and Deep Learning
https://people.idsia.ch/~juergen/deep-learning-history.html
https://arxiv.org/abs/2212.11279
1. The AI100 report starts with 1956 when the term "AI" was coined by John McCarthy. However, practical AI research dates back at least to 1914, when Leonardo Torres y Quevedo built the first working chess end game player [BRU1-4] (back then chess was considered as an activity restricted to the realms of intelligent creatures). The machine was still considered impressive decades later when another AI pioneer—Norbert Wiener [WI48]—played against it at the 1951 Paris AI conference [AI51][BRO21][BRU4].
AI theory dates back at least to 1931-34 when Kurt Gödel (see below) identified fundamental limits of any type of computation-based AI [GOD][BIB3][GOD21,a,b].
2. The AI100 report mentions Bayesian reasoning which is actually Laplacian or possibly Saundersonian [STI83-85] reasoning [BAY1-8][FI22].
3. The AI100 report mentions the Boolean Algebra (1847) [BOO], but not Leibniz' much earlier formal Algebra of Thought (1686)[L86][WI48] which is deductively equivalent [LE18].
4. The AI100 report mentions Babbage's ideas on computing, but not the first commercial program-controlled machines (punch card-based looms) built much earlier in France circa 1800 by Jacquard and others - perhaps the first "modern" programmers who wrote the world's first industrial software. (Babbage planned but was unable to build a programmable, general purpose computer - only his non-universal special purpose calculator led to a working 20th century replica.)
5. The AI100 report claims: "The most influential ideas underpinning computer science came from Alan Turing, who proposed a formal model of computing.”
However, the theory of AI and computer science was not founded by Turing but by Kurt Gödel (1931-34) [GOD][GOD34] who identified the fundamental limits of algorithmic theorem proving, computing, and any type of computation-based AI [GOD][BIB3], as well as computer science's most famous open problem "P=NP?” [GOD56][URQ10] (there is a reason why there is a Gödel Prize for theoretical computer science). Gödel's 1931 work was extended by Alonzo Church, whose 1935 result [CHU] on the decision problem preceded Turing's identical 1936 result [TUR].
Turing's 1936 paper [TUR] was claimed to provide the "theoretical backbone" for all computers to come [NASC7]. However, it just described a simple theoretical and impractical pen & paper construct like those of Gödel (1931-34) [GOD][GOD34], Church (1935) [CHU] and Post (1936) [POS] which did not even feature elementary practical building blocks such as addressable memory. It was actually Konrad Zuse's 1936 patent application [ZU36] which really described what became the first practical general-purpose program-controlled computer (completed in 1941) [ZUS21,a,b][RO98].
Unlike Church (1935) [CHU], Turing (1936) did not even consider halting programs [TUR]; the famous halting problem (sometimes attributed to him) was actually named by Davis in 1958 [HLT58] and formulated by Kleene in 1952 [HLT52][HLT21].
The famous "Turing Test” (mentioned by the AI100 report) was actually predefined by Descartes [TUR3,a,b].
6. The AI100 report mentions Newell and Simon's Logic Theorist program (1956)[NS56], but ignores the original work on automatic theorem proving by Zuse (1948) [ZU48].
7. The AI100 report claims that "Rosenblatt's Perceptron [154], a computational model based on biological neurons, became the basis for the field of artificial neural networks." However, already in 1805, Legendre published what's now called a linear neural network (NN). Later Gauss was also credited for earlier unpublished work on this done circa 1795 [STI81]. Of course, back then this was not called an NN. It was called the method of least squares, also widely known as linear regression. But it is mathematically identical to today's linear NNs: same basic algorithm, same error function, same adaptive parameters/weights. In fact, many NN courses start by introducing this method, then move on to more complex, deeper NNs.
Rosenblatt, however, not only combined linear NNs and threshold functions, he also had more interesting, deeper multilayer perceptrons (MLPs) [R58]. His MLPs had a non-learning first layer with randomized weights and an adaptive output layer. Although this was not yet "deep learning," because only the last layer learned [DL1], Rosenblatt basically had what much later was rebranded as Extreme Learning Machines (ELMs) without proper attribution [ELM1-2][CONN21][T22].
Successful deep learning in _deep_ feedforward network architectures started in 1965 in the Ukraine when Alexey Ivakhnenko & Valentin Lapa introduced the first general, working learning algorithms for deep MLPs with arbitrarily many hidden layers (already containing the now popular multiplicative gates) [DEEP1-2][DL1-2][FDL].
And in 1967-68, Shun-Ichi Amari trained MLPs with many layers in non-incremental end-to-end fashion from scratch by stochastic gradient descent (SGD) [GD1], a method proposed in 1951 by Robbins & Monro [STO51-52].
So much for now!
Much more (including 555+ partially annotated references) in the
Annotated History of Modern AI and Deep Learning
https://people.idsia.ch/~juergen/deep-learning-history.html
https://arxiv.org/abs/2212.11279
Juergen
Prof. Jürgen Schmidhuber
Director, AI Initiative, KAUST
Scientific Director, Swiss AI Lab IDSIA
Co-Founder & Chief Scientist, NNAISENSE
http://www.idsia.ch/~juergen/blog.html
> --
> You received this message because you are subscribed to the Google Groups "Machine Learning News" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ml-news+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/ml-news/14202.1675028352%40cs.utexas.edu.
I read the "Appendix I: A Short History of AI (Annotated)” of the AI100 report. It starts with a disclaimer: "A complete and fully balanced history of the field is beyond the scope of this document.”
In fact, it almost completely fails to credit the true pioneers of practical AI (including deep neural nets and computational hardware) and AI theory. Let me try to correct some of the most misleading statements. All references below can be found in the 2022 survey:
Annotated History of Modern AI and Deep Learning
https://people.idsia.ch/~juergen/deep-learning-history.html
https://arxiv.org/abs/2212.11279
1. The AI100 report starts with 1956 when the term "AI" was coined by John McCarthy. However, practical AI research dates back at least to 1914, when Leonardo Torres y Quevedo built the first working chess end game player [BRU1-4] (back then chess was considered as an activity restricted to the realms of intelligent creatures). The machine was still considered impressive decades later when another AI pioneer—Norbert Wiener [WI48]—played against it at the 1951 Paris AI conference [AI51][BRO21][BRU4].
AI theory dates back at least to 1931-34 when Kurt Gödel (see below) identified fundamental limits of any type of computation-based AI [GOD][BIB3][GOD21,a,b].
2. The AI100 report mentions Bayesian reasoning which is actually Laplacian or possibly Saundersonian [STI83-85] reasoning [BAY1-8][FI22].
3. The AI100 report mentions the Boolean Algebra (1847) [BOO], but not Leibniz' much earlier formal Algebra of Thought (1686)[L86][WI48] which is deductively equivalent [LE18].
4. The AI100 report mentions Babbage's ideas on computing, but not the first commercial program-controlled machines (punch card-based looms) built much earlier in France circa 1800 by Jacquard and others - perhaps the first "modern" programmers who wrote the world's first industrial software. (Babbage planned but was unable to build a programmable, general purpose computer - only his non-universal special purpose calculator led to a working 20th century replica.)
5. The AI100 report claims: "The most influential ideas underpinning computer science came from Alan Turing, who proposed a formal model of computing.”
However, the theory of AI and computer science was not founded by Turing but by Kurt Gödel (1931-34) [GOD][GOD34] who identified the fundamental limits of algorithmic theorem proving, computing, and any type of computation-based AI [GOD][BIB3], as well as computer science's most famous open problem "P=NP?” [GOD56][URQ10] (there is a reason why there is a Gödel Prize for theoretical computer science). Gödel's 1931 work was extended by Alonzo Church, whose 1935 result [CHU] on the decision problem preceded Turing's identical 1936 result [TUR].
Turing's 1936 paper [TUR] was claimed to provide the "theoretical backbone" for all computers to come [NASC7]. However, it just described a simple theoretical and impractical pen & paper construct like those of Gödel (1931-34) [GOD][GOD34], Church (1935) [CHU] and Post (1936) [POS] which did not even feature elementary practical building blocks such as addressable memory. It was actually Konrad Zuse's 1936 patent application [ZU36] which really described what became the first practical general-purpose program-controlled computer (completed in 1941) [ZUS21,a,b][RO98].
Unlike Church (1935) [CHU], Turing (1936) did not even consider halting programs [TUR]; the famous halting problem (sometimes attributed to him) was actually named by Davis in 1958 [HLT58] and formulated by Kleene in 1952 [HLT52][HLT21].
The famous "Turing Test” (mentioned by the AI100 report) was actually predefined by Descartes [TUR3,a,b].
6. The AI100 report mentions Newell and Simon's Logic Theorist program (1956)[NS56], but ignores the original work on automatic theorem proving by Zuse (1948) [ZU48].
7. The AI100 report claims that "Rosenblatt's Perceptron [154], a computational model based on biological neurons, became the basis for the field of artificial neural networks." However, already in 1805, Legendre published what's now called a linear neural network (NN). Later Gauss was also credited for earlier unpublished work on this done circa 1795 [STI81]. Of course, back then this was not called an NN. It was called the method of least squares, also widely known as linear regression. But it is mathematically identical to today's linear NNs: same basic algorithm, same error function, same adaptive parameters/weights. In fact, many NN courses start by introducing this method, then move on to more complex, deeper NNs.
Rosenblatt, however, not only combined linear NNs and threshold functions, he also had more interesting, deeper multilayer perceptrons (MLPs) [R58]. His MLPs had a non-learning first layer with randomized weights and an adaptive output layer. Although this was not yet "deep learning," because only the last layer learned [DL1], Rosenblatt basically had what much later was rebranded as Extreme Learning Machines (ELMs) without proper attribution [ELM1-2][CONN21][T22].
Successful deep learning in _deep_ feedforward network architectures started in 1965 in the Ukraine when Alexey Ivakhnenko & Valentin Lapa introduced the first general, working learning algorithms for deep MLPs with arbitrarily many hidden layers (already containing the now popular multiplicative gates) [DEEP1-2][DL1-2][FDL].
And in 1967-68, Shun-Ichi Amari trained MLPs with many layers in non-incremental end-to-end fashion from scratch by stochastic gradient descent (SGD) [GD1], a method proposed in 1951 by Robbins & Monro [STO51-52].
So much for now!
Much more (including 555+ partially annotated references) in the
Annotated History of Modern AI and Deep Learning
https://people.idsia.ch/~juergen/deep-learning-history.html
https://arxiv.org/abs/2212.11279
Juergen
Prof. Jürgen Schmidhuber
Director, AI Initiative, KAUST
Scientific Director, Swiss AI Lab IDSIA
Co-Founder & Chief Scientist, NNAISENSE
http://www.idsia.ch/~juergen/blog.html
> You received this message because you are subscribed to the Google Groups "Machine Learning News" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ml-news+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/ml-news/14202.1675028352%40cs.utexas.edu.
Reply all
Reply to author
Forward
0 new messages