ITS seminar, Friday (1/24) 11am
0 views
Skip to first unread message
Tankut Can
Jan 22, 2020, 6:12:09 PM1/22/20
to condmatcunyGC
Hi all,
We are pleased to have Jeffrey Pennington speaking at the ITS seminar this Friday (1/24) at 11am in GC 5209. Title and abstract appear below. Hope to see you there.
Title:
Deep learning and operator-valued free probability: training and generalization dynamics in high dimensions
Abstract:
One of the distinguishing characteristics of modern deep learning systems is that they typically employ neural network architectures that utilize enormous numbers of parameters, often in the millions and sometimes even in the billions. While this paradigm has recently inspired a broad research effort on the properties of large networks, relatively little work has been devoted to the fact that these networks are often used to model large complex datasets, which may themselves contain millions or even billions of constraints. In this talk, I will present a formalism based on operator-valued free probability that enables exact predictions of training and generalization performance in the high-dimensional regime in which both the dataset size and the number of features tend to infinity. The analysis provides one of the first analytically tractable models that captures the effects of early stopping, over/under-parameterization, explicit regularization, and which exhibits the characteristic double-descent curve.
Abstract:
One of the distinguishing characteristics of modern deep learning systems is that they typically employ neural network architectures that utilize enormous numbers of parameters, often in the millions and sometimes even in the billions. While this paradigm has recently inspired a broad research effort on the properties of large networks, relatively little work has been devoted to the fact that these networks are often used to model large complex datasets, which may themselves contain millions or even billions of constraints. In this talk, I will present a formalism based on operator-valued free probability that enables exact predictions of training and generalization performance in the high-dimensional regime in which both the dataset size and the number of features tend to infinity. The analysis provides one of the first analytically tractable models that captures the effects of early stopping, over/under-parameterization, explicit regularization, and which exhibits the characteristic double-descent curve.
Reply all
Reply to author
Forward
0 new messages