OxCSML seminars this week: Arno Solin and Stathi Fotiadis
24 views
Skip to first unread message
Hai Dang Dau
Oct 30, 2023, 4:27:40 PM10/30/23
to oxc...@googlegroups.com, oxcsml...@googlegroups.com
Dear all,
This week, we welcome two speakers to our OxCSML seminar: Arno
Solin and Stathi Fotiadis. Please see details of their talks below. Do
note that they are on different dates and take place in different rooms.
We were aware of participants having problems connecting to Zoom
last week, sorry for that! Everything should be resolved for now; if you
still experience any difficulty please email us directly.
Looking forwards to seeing you there,
Kind regards,
Saif & Hai-Dang.
==================
Speaker: Arno Solin, Aalto University
Time and date: 15:30 to 16.30 Thursday 2 November
Place: Room Large lecture theatre, Department of Statistics, Oxford
Zoom: https://zoom.us/j/94334081927?pwd=ZGZWRmJJUVhjMndkT1dNQnpuOFVrdz09
Meeting ID: 943 3408 1927
Passcode: 118389
Title:
Structured Inductive Biases in Machine Learning: Approaches to Magnetic SLAM and Multi-scale Generative Modelling
Abstract:
This talk considers two mutually different application areas in machine learning that benefit from inducing structure in the problem formulation: Magnetic simultaneous localization and mapping (SLAM) and generative modelling of images with diffusion models (see [1] and [2], respectively). Magnetic SLAM relies on particle filtering (sequential Monte Carlo) to learn a map of the ambient magnetic field and simultaneously use it for localization/tracking. A key component in such a SLAM system is aiding the magnetic vector-field mapping process with a Gaussian process prior to encoding physical constraints from Maxwell's equations. The second part of the talk considers the innate multi-scale structure within images as part of generative modelling. Drawing inspiration from both diffusion models and the empirical success of coarse-to-fine strategies, this talk presents a recent approach that mimics the reverse of the heat equation. This partial differential equation, when applied to a 2D image plane, inherently diminishes fine-scale details. Our method infers the forward heat equation combined with consistent additive noise as a variational approximation to the diffusion latent variable model. In the broader scope, this talk aims to view model-induced bias/structure as a potential strength instead of a possible weakness.
[1] Manon Kok, Arno Solin, and Thomas B. Schön (2023). Rao-Blackwellized Particle Smoothing for Simultaneous Localization and Mapping. pre-print: https://arxiv.org/abs/2306.03953
[2] Severi Rissanen, Markus Heinonen, and Arno Solin (2023). Generative modelling with inverse heat dissipation. In International Conference on Learning Representations (ICLR). pre-print: https://arxiv.org/abs/2206.13397
Short bio:
Dr. Arno Solin is an Assistant Professor (tenure-track) in Machine Learning at the Department of Computer Science at Aalto University, Finland, and an ELLIS Scholar. His research interests are in data-efficient machine learning, with a particular interest in probabilistic methods for sequential models, real-time inference, and sensor fusion. For more information, see his homepage (https://arno.solin.fi/) or Google Scholar profile
Time and date: 15:30 to 16.30 Thursday 2 November
Place: Room Large lecture theatre, Department of Statistics, Oxford
Zoom: https://zoom.us/j/94334081927?pwd=ZGZWRmJJUVhjMndkT1dNQnpuOFVrdz09
Meeting ID: 943 3408 1927
Passcode: 118389
Title:
Structured Inductive Biases in Machine Learning: Approaches to Magnetic SLAM and Multi-scale Generative Modelling
Abstract:
This talk considers two mutually different application areas in machine learning that benefit from inducing structure in the problem formulation: Magnetic simultaneous localization and mapping (SLAM) and generative modelling of images with diffusion models (see [1] and [2], respectively). Magnetic SLAM relies on particle filtering (sequential Monte Carlo) to learn a map of the ambient magnetic field and simultaneously use it for localization/tracking. A key component in such a SLAM system is aiding the magnetic vector-field mapping process with a Gaussian process prior to encoding physical constraints from Maxwell's equations. The second part of the talk considers the innate multi-scale structure within images as part of generative modelling. Drawing inspiration from both diffusion models and the empirical success of coarse-to-fine strategies, this talk presents a recent approach that mimics the reverse of the heat equation. This partial differential equation, when applied to a 2D image plane, inherently diminishes fine-scale details. Our method infers the forward heat equation combined with consistent additive noise as a variational approximation to the diffusion latent variable model. In the broader scope, this talk aims to view model-induced bias/structure as a potential strength instead of a possible weakness.
[1] Manon Kok, Arno Solin, and Thomas B. Schön (2023). Rao-Blackwellized Particle Smoothing for Simultaneous Localization and Mapping. pre-print: https://arxiv.org/abs/2306.03953
[2] Severi Rissanen, Markus Heinonen, and Arno Solin (2023). Generative modelling with inverse heat dissipation. In International Conference on Learning Representations (ICLR). pre-print: https://arxiv.org/abs/2206.13397
Short bio:
Dr. Arno Solin is an Assistant Professor (tenure-track) in Machine Learning at the Department of Computer Science at Aalto University, Finland, and an ELLIS Scholar. His research interests are in data-efficient machine learning, with a particular interest in probabilistic methods for sequential models, real-time inference, and sensor fusion. For more information, see his homepage (https://arno.solin.fi/) or Google Scholar profile
===================
Speaker: Stathi Fotiadis (Imperial Colledge London)
Time and date: 14.00 to 15.00 Friday 3 November
Place: Room LG.03 (Small lecture theatre), Department of Statistics, Oxford
Zoom: https://zoom.us/j/92566824256?pwd=TE84cFBUOXVNRFQ1R2dLL25mYmo0UT09
Meeting ID: 925 6682 4256
Passcode: 502555
Title:
Accelerating Diffusion Model Sampling: Techniques for Training and Inference
Summary:
In this seminar, I will present two techniques, used during training and inference respectively, to accelerate the sampling of diffusion models for image generation. The primary focus is on Shortest Path Diffusion (SPD), a method rooted on the hypothesis that the optimal image corruption schedule minimizes the length between two distributions. By utilizing the Fisher metric in the space of probability distributions, SPD introduces a unique non-uniform diffusion schedule that performs image sharpening and noise deblurring. Results indicate that SPD-trained models outperform the image quality of previous schedules on various datasets.
I will also discuss an improvement that can be used at inference-time and builds upon the Diffusion Exponential Integrator Sampler (DEIS). DEIS takes advantage of the semi-linear nature of the probability flow ODE to minimize integration error during sample generation. An important aspect of this approach is the reparameterization of the score function. DEIS-SN proposes a new score parameterization that smooths out the rapid changes at the end of the sampling process leading to significant improvements in image quality. Using publicly available models pretrained on CIFAR-10, DEIS-SN achieves new state-of-the-art FID results at very low number of Neural Function Evaluations(NFEs).
Time and date: 14.00 to 15.00 Friday 3 November
Place: Room LG.03 (Small lecture theatre), Department of Statistics, Oxford
Zoom: https://zoom.us/j/92566824256?pwd=TE84cFBUOXVNRFQ1R2dLL25mYmo0UT09
Meeting ID: 925 6682 4256
Passcode: 502555
Title:
Accelerating Diffusion Model Sampling: Techniques for Training and Inference
Summary:
In this seminar, I will present two techniques, used during training and inference respectively, to accelerate the sampling of diffusion models for image generation. The primary focus is on Shortest Path Diffusion (SPD), a method rooted on the hypothesis that the optimal image corruption schedule minimizes the length between two distributions. By utilizing the Fisher metric in the space of probability distributions, SPD introduces a unique non-uniform diffusion schedule that performs image sharpening and noise deblurring. Results indicate that SPD-trained models outperform the image quality of previous schedules on various datasets.
I will also discuss an improvement that can be used at inference-time and builds upon the Diffusion Exponential Integrator Sampler (DEIS). DEIS takes advantage of the semi-linear nature of the probability flow ODE to minimize integration error during sample generation. An important aspect of this approach is the reparameterization of the score function. DEIS-SN proposes a new score parameterization that smooths out the rapid changes at the end of the sampling process leading to significant improvements in image quality. Using publicly available models pretrained on CIFAR-10, DEIS-SN achieves new state-of-the-art FID results at very low number of Neural Function Evaluations(NFEs).
Reply all
Reply to author
Forward
0 new messages