Daily TMLR digest for Aug 12, 2022

0 views
Skip to first unread message

TMLR

unread,
Aug 11, 2022, 8:00:07 PM8/11/22
to tmlr-anno...@googlegroups.com

Accepted papers
===============


Title: Mean-Field Langevin Dynamics : Exponential Convergence and Annealing

Authors: Lénaïc Chizat

Abstract: Noisy particle gradient descent (NPGD) is an algorithm to minimize convex functions over the space of measures that include an entropy term. In the many-particle limit, this algorithm is described by a Mean-Field Langevin dynamics---a generalization of the Langevin dynamic with a non-linear drift---which is our main object of study. Previous work have shown its convergence to the unique minimizer via non-quantitative arguments. We prove that this dynamics converges at an exponential rate, under the assumption that a certain family of Log-Sobolev inequalities holds. This assumption holds for instance for the minimization of the risk of certain two-layer neural networks, where NPGD is equivalent to standard noisy gradient descent. We also study the annealed dynamics, and show that for a noise decaying at a logarithmic rate, the dynamics converges in value to the global minimizer of the unregularized objective function.

URL: https://openreview.net/forum?id=BDqzLH1gEm

---

Title: Variational Disentanglement for Domain Generalization

Authors: Yufei Wang, Haoliang Li, Hao Cheng, Bihan Wen, Lap-Pui Chau, Alex Kot

Abstract: Domain generalization aims to learn a domain-invariant model that can generalize well to the unseen target domain. In this paper, based on the assumption that there exists an invariant feature mapping, we propose an evidence upper bound of the divergence between the category-specific feature and its invariant ground-truth using variational inference. To optimize this upper bound, we further propose an efficient Variational Disentanglement Network (VDN) that is capable of disentangling the domain-specific features and category-specific features (which generalize well to the unseen samples). Besides, the generated novel images from VDN are used to further improve the generalization ability. We conduct extensive experiments to verify our method on three benchmarks, and both quantitative and qualitative results illustrate the effectiveness of our method.

URL: https://openreview.net/forum?id=fudOtITMIZ

---

Title: On Robustness to Missing Video for Audiovisual Speech Recognition

Authors: Oscar Chang, Otavio Braga, Hank Liao, Dmitriy Serdyuk, Olivier Siohan

Abstract: It has been shown that learning audiovisual features can lead to improved speech recognition performance over audio-only features, especially for noisy speech. However, in many common applications, the visual features are partially or entirely missing, e.g. the speaker might move off screen. Multi-modal models need to be robust: missing video frames should not degrade the performance of an audiovisual model to be worse than that of a single-modality audio-only model. While there have been many attempts at building robust models, there is little consensus on how robustness should be evaluated. To address this, we introduce a framework that allows claims about robustness to be evaluated in a precise and testable way. We also conduct a systematic empirical study of the robustness of common audiovisual speech recognition architectures on a range of acoustic noise conditions and test suites. Finally, we show that an architecture-agnostic solution based on cascades can consistently achieve robustness to missing video, even in settings where existing techniques for robustness like dropout fall short.

URL: https://openreview.net/forum?id=fXorxxbDvO

---


New submissions
===============


Title: Margin based Self-Supervised Neural Architecture Search

Abstract: Neural Architecture Search (NAS) has been used recently to achieve improved performance in various tasks and most prominently in image classification. Yet, most search strategies rely on large labeled datasets, which limit their usage in the case where only a smaller fraction of the data is annotated. Self-supervised learning has shown great promise in training neural networks using unlabeled data. In this work, we propose a self-supervised neural architecture search (SSNAS) that allows finding novel network models without the need for labeled data. We show that such a search leads to comparable results to supervised training with a ``fully labeled'' NAS. While such a result has been shown in concurrent works, the uniqueness of this work is that we also show that such a search can also improve the performance of self-supervised learning. We show that using the learned architectures for self-supervised representation learning leads to improved performance. Thus, SSL can both improve NAS and be improved by it.
Specifically, due to the common case of resource constrains, we exhibit the advantage of our approach when the number of labels in the search is relatively small.

URL: https://openreview.net/forum?id=qSnHmZg63d

---

Title: An Attract-Repel Decomposition of Undirected Networks

Abstract: Dot product latent space models are a standard method in many areas ranging from social network analysis to computational biology. Such models have issues modeling graphs which include unclosed triangles such as social networks which include latent heterophily (i.e. cases where opposites attract) or co-occurrence graphs which have substitutes (items which occur in similar contexts but not together). We show a minimal expansion to the dot product model which includes both homophily (attract) and heterophily (repel) latent forces. Beyond simply fitting the data, we discuss how to use the AR spaces produced to more deeply understand real networks allowing analysts to measure the latent heterophily in social network formation, detect substitutes in co-occurrence networks, or perform exploratory analysis for candidates for inhibition / activation relationships in systems biology.


URL: https://openreview.net/forum?id=P18uCQqEcb

---

Reply all
Reply to author
Forward
0 new messages