New Paper on "the deep history of deep learning"

[Stephen José Hanson]

lease find a new paper on Deep Learning, its history and connections
with Psychological Learning Theory from 100 years ago.

This is a sketch of how Deep Learning works and why. This paper
reviews relevant literature and provides testable hypothesis
concerning the development of feature structures in large networks and
how they develop and are curated.

Even now we don't understand how deep learning works or why it applies
so generally across many kinds of domain. It generality is more of a
mystery when one considers the feature construction process. This of
course is even worse for LLMs which noone has a clue how they come to
talk, or seems to have executive functions. And yet it is being
monetized and exploited on a daily basis (not new in the history of
technology). Yoshua Bengio, last year at the introduction of an
LLM summer school, said something quite prescient: "I believe we are
all sleep walking through this".

This paper is an attempt to look under the hood and make sense of the
generality of deep learning and the AI it enabled.


L.L. Thurstone, The law of effect and the dynamics of deep learning “a
deep history of deep learning”

SJ Hanson and C Hanson

ABSTRACT

Deep learning (DL), a variant of the neural network algorithms
originally proposed in the early twentieth century, has resulted in a
renaissance of artificial intelligence. Despite the growing dominance
of DL networks, little is understood about the learning mechanism that
makes these networks so effective across such a wide range of
applications. Drawing on a century of psychological learning theory
(e.g., LL Thurstone), an account is offered of the learning mechanism
that may enable DL networks to perform so successfully across so many
different tasks. Specifically, evidence is provided that learning in
DL networks is fit best by a hyperbolic function. This function is
independent of hyper/meta parameters–not a scaling function but a
learning curve to a specific equilibrium, a function that also entails
an autocatalytic mechanism through which complex structures,
abstracted from sensory features, can in principle create and support
cognitive function in both biological and artificial systems.
Keywords: Learning theory, Deep Learning, hyperbolic, autocatalytic,
AI, Thurstone


https://link.springer.com/article/10.1007/s10462-026-11560-3