Reminder: Tomorrow's CV-DL Seminar – Assaf Shocher | Teaching Neural Networks Linear Algebra
3 views
Skip to first unread message
Naaman Kopty
May 9, 2026, 1:55:06 PMMay 9
to 'Google Groups' via CV-DL Seminar – University of Haifa
Dear all,
You are invited to attend the next talk in the CV-DL Seminar Series.
Assaf Shocher
Technion
Technion
Sunday, May 10, 2026
11:15-12:30
Room 508, Amir Building
11:15-12:30
Room 508, Amir Building
Title:
Teaching Neural Networks Linear Algebra
Teaching Neural Networks Linear Algebra
Abstract:
Neural networks are powerful but notoriously difficult to analyze, compose, or control. Linear algebra, by contrast, is the mathematical ideal of tractability. In this talk I will present several works from my lab that aim to import the principles of linear algebra into deep learning. I begin with projection as a generative principle: Idempotent Generative Networks (IGN) train a neural network to satisfy f(f(z)) = f(z), so that the data manifold emerges as the set of fixed points of the operator. Generation then becomes projection: a single forward pass maps noise to the manifold, while repeated application enables principled refinement. We then ask a more provocative question: "Who said neural networks aren't linear?". Neural networks are famously nonlinear, but nonlinear with respect to which vector spaces? Using the algebraic notion of transport of structure, the Linearizer framework identifies non-standard vector spaces in which a neural network acts as a linear operator. In these spaces, tools such as SVD, pseudo-inverses, and composition become directly applicable to neural networks, with consequences ranging from algebraic analysis of models to collapsing diffusion sampling into a single step. Finally, I will present recent work that generalizes the Moore-Penrose pseudo-inverse to nonlinear mappings. Surjective Pseudo-invertible Neural Networks (SPNN) satisfy the classical Penrose identities by construction, enabling nonlinear back-projection and extending diffusion-based zero-shot inverse problem solving from linear degradations to arbitrary nonlinear ones. These are steps we are taking towards systems that we can study and design with the same rigor and elegance that linear algebra brings to the physical sciences.
Neural networks are powerful but notoriously difficult to analyze, compose, or control. Linear algebra, by contrast, is the mathematical ideal of tractability. In this talk I will present several works from my lab that aim to import the principles of linear algebra into deep learning. I begin with projection as a generative principle: Idempotent Generative Networks (IGN) train a neural network to satisfy f(f(z)) = f(z), so that the data manifold emerges as the set of fixed points of the operator. Generation then becomes projection: a single forward pass maps noise to the manifold, while repeated application enables principled refinement. We then ask a more provocative question: "Who said neural networks aren't linear?". Neural networks are famously nonlinear, but nonlinear with respect to which vector spaces? Using the algebraic notion of transport of structure, the Linearizer framework identifies non-standard vector spaces in which a neural network acts as a linear operator. In these spaces, tools such as SVD, pseudo-inverses, and composition become directly applicable to neural networks, with consequences ranging from algebraic analysis of models to collapsing diffusion sampling into a single step. Finally, I will present recent work that generalizes the Moore-Penrose pseudo-inverse to nonlinear mappings. Surjective Pseudo-invertible Neural Networks (SPNN) satisfy the classical Penrose identities by construction, enabling nonlinear back-projection and extending diffusion-based zero-shot inverse problem solving from linear degradations to arbitrary nonlinear ones. These are steps we are taking towards systems that we can study and design with the same rigor and elegance that linear algebra brings to the physical sciences.
Relevant papers:
https://assafshocher.github.io/linearizer/
https://assafshocher.github.io/IGN/
https://yamitehr.github.io/spnn-project-page/
https://www.norange.io/projects/ittt/
https://assafshocher.github.io/linearizer/
https://assafshocher.github.io/IGN/
https://yamitehr.github.io/spnn-project-page/
https://www.norange.io/projects/ittt/
Bio:
I am an Assistant Professor at the Technion’s Faculty of Data and Decision Sciences. Previously, I was a researcher at NVIDIA. Prior to that I was a postdoctoral fellow at UC Berkeley working with Alyosha Efros and a visiting researcher at Google. I received my PhD from the Weizmann Institute of Science, where I was advised by Michal Irani, and hold bachelor’s degrees in Physics and Electrical Engineering from Ben-Gurion University. My prizes and honors include the Chaya carrier advancement chair, Rothschild postdoctoral fellowship, the Fulbright postdoctoral fellowship, the John F. Kennedy award for outstanding Ph.D. at the Weizmann Institute, and the Blavatnik award for CS Ph.D. graduates.
I am an Assistant Professor at the Technion’s Faculty of Data and Decision Sciences. Previously, I was a researcher at NVIDIA. Prior to that I was a postdoctoral fellow at UC Berkeley working with Alyosha Efros and a visiting researcher at Google. I received my PhD from the Weizmann Institute of Science, where I was advised by Michal Irani, and hold bachelor’s degrees in Physics and Electrical Engineering from Ben-Gurion University. My prizes and honors include the Chaya carrier advancement chair, Rothschild postdoctoral fellowship, the Fulbright postdoctoral fellowship, the John F. Kennedy award for outstanding Ph.D. at the Weizmann Institute, and the Blavatnik award for CS Ph.D. graduates.
We look forward to seeing you there.
Reply all
Reply to author
Forward
0 new messages