Weekly TMLR digest for Jul 13, 2025
TMLR
New certifications
==================
Featured Certification: Task Arithmetic Through The Lens Of One-Shot Federated Learning
Zhixu Tao, Ian Mason, Sanjeev Kulkarni, Xavier Boix
https://openreview.net/forum?id=Cgyo7S7Oy0
---
Expert Certification: Gaussian Processes with Bayesian Inference of Covariate Couplings
Mattia Rosso, Juho Ylä-Jääski, Zheyang Shen, Markus Heinonen, Maurizio Filippone
https://openreview.net/forum?id=fameEAljo3
---
Accepted papers
===============
Title: Towards Efficient Contrastive PAC Learning
Authors: Jie Shen
Abstract: We study contrastive learning under the PAC learning framework. While a series of recent works have shown statistical results for learning under contrastive loss, based either on the VC-dimension or Rademacher complexity, their algorithms are inherently inefficient or not implying PAC guarantees. In this paper, we consider contrastive learning of the fundamental concept of linear representations. Surprisingly, even under such basic setting, the existence of efficient PAC learners is largely open. We first show that the problem of contrastive PAC learning of linear representations is intractable to solve in general. We then show that it can be relaxed to a semi-definite program when the distance between contrastive samples is measured by the $\ell_2$-norm. We then establish generalization guarantees based on Rademacher complexity, and connect it to PAC guarantees under certain contrastive large-margin conditions. To the best of our knowledge, this is the first efficient PAC learning algorithm for contrastive learning.
URL: https://openreview.net/forum?id=dBJo9hyKVg
---
Title: FB-MOAC: A Reinforcement Learning Algorithm for Forward-Backward Markov Decision Processes
Authors: Mohsen Amidzadeh, Mario Di Francesco
Abstract: Reinforcement learning (RL) algorithms are effective in solving problems that can be modeled as Markov decision processes (MDPs). These algorithms primarily target forward MDPs whose dynamics evolve over time from an initial state. However, several important problems in different scenarios including stochastic control and network systems exhibit both a forward and a backward dynamics. As a consequence, they cannot be expressed as a standard MDP, thereby calling for a novel theory for RL in this context. Accordingly, this work introduces the concept of Forward-Backward Markov Decision Processes (FB-MDPs) for multi-objective problems, develops a novel theoretical framework to characterize their optimal solutions, and proposes a general forward-backward step-wise template that allows to adapt RL algorithms for FB-MDP problems. A Forward Backward Multi Objective Actor Critic (FB-MOAC) algorithm is introduced accordingly to obtain optimal policies with guaranteed convergence and a competitive rate with respect to standard approaches in RL. FB-MOAC is evaluated on diverse use cases in the context of mathematical finance and mobile resource management. The obtained results show that FB-MOAC outperforms the state of the art across different metrics, highlighting its ability to learn and maximize rewards.
URL: https://openreview.net/forum?id=li5DyC6rfS
---
Title: A note on the $k$-means clustering for missing data
Authors: Yoshikazu Terada, Xin Guan
Abstract: The classical $k$-means clustering algorithm requires complete data and cannot be directly applied when observations contain missing entries. An intuitive and computationally efficient extension addresses this issue by minimizing the $k$-means loss over the observed entries only, a strategy considered in several studies. This method is known as $k$-POD clustering. In this paper, we provide a theoretical analysis of this approach and demonstrate that it is generally inconsistent, even under the missing completely at random (MCAR) assumption. Specifically, we show that the expected loss being minimized asymptotically differs from the original $k$-means objective, leading to biased estimates of cluster centers in the large-sample limit. This highlights a fundamental limitation: the method may fail to recover the true underlying cluster structure, even in settings where $k$-means performs well on fully observed data. Nevertheless, when the missing rate per variable is sufficiently low and the dimensionality is high, the method can still produce stable and practically useful results, making it a viable alternative when the complete-case analysis is ineffective.
URL: https://openreview.net/forum?id=pcqlTvePXS
---
Title: Synthesizing world models for bilevel planning
Authors: Zergham Ahmed, Joshua B. Tenenbaum, Chris Bates, Samuel J. Gershman
Abstract: Modern reinforcement learning (RL) systems have demonstrated remarkable capabilities in complex environments, such as video games. However, they still fall short of achieving human-like sample efficiency and adaptability when learning new domains. Theory-based reinforcement learning (TBRL) is an algorithmic framework specifically designed to address this gap. Modeled on cognitive theories, TBRL leverages structured, causal world models---``theories''---as forward simulators for use in planning, generalization and exploration. Although current TBRL systems provide compelling explanations of how humans learn to play video games, they face several technical limitations: their theory languages are restrictive, and their planning algorithms are not scalable. To address these challenges, we introduce TheoryCoder, an instantiation of TBRL that exploits hierarchical representations of theories and efficient program synthesis methods for more powerful learning and planning. TheoryCoder equips agents with general-purpose abstractions (e.g., ``move to''), which are then grounded in a particular environment by learning a low-level transition model (a Python program synthesized from observations by a large language model). A bilevel planning algorithm can exploit this hierarchical structure to solve large domains. We demonstrate that this approach can be successfully applied to diverse and challenging grid-world games, where approaches based on directly synthesizing a policy perform poorly. Ablation studies demonstrate the benefits of using hierarchical abstractions.
URL: https://openreview.net/forum?id=m9V4JHLJrD
---
Title: Federated Generalized Novel Category Discovery with Prompts Tuning
Authors: Lei Shen, Nan Pu, Zhun Zhong, Mingming Gong, Dianhai Yu, Chengqi Zhang, Bo Han
Abstract: Generalized category discovery (GCD) is proposed to handle categories from unseen labels during the inference stage by clustering them. Most works in GCD provide solutions for unseen classes in data-centralized settings. However, unlabeled categories possessed by clients, which are common in real-world federated learning (FL), have been largely ignored and degraded the performance of classic FL algorithms. To demonstrate and mitigate the harmful effect of unseen classes, we dive into a GCD problem setting applicable for FL named FedGCD, analyze overfitting problem in FedGCD in detail, establish a strong baseline constructed with state-of-the-art GCD algorithm simGCD, and design a learning framework with prompt tuning to tackle both the overfitting and communication burden problems in FedGCD. In our methods, clients first separately carry out prompt learning on local data. Then, we aggregate the prompts from all clients as the global prompt to help capture global knowledge and then send the global prompts to local clients to allow access to broader knowledge from other clients. By this method, we significantly reduce the parameters needed to upload in FedGCD, which is a common obstacle in the real application of most FL algorithms. We conduct experiments on both generic and fine-grained datasets like CIFAR-100 and CUB-200, and show that our method is comparable to the FL version of simGCD and surpasses other baselines with significantly fewer parameters to transmit.
URL: https://openreview.net/forum?id=dVMESwnMlo
---
Title: Finetuning CLIP to Reason about Pairwise Differences
Authors: Dylan Sam, Devin Willmott, João D. Semedo, J Zico Kolter
Abstract: Vision-language models (VLMs) such as CLIP are trained via contrastive learning between text and image pairs, resulting in aligned image and text embeddings that are useful for many downstream tasks. A notable drawback of CLIP, however, is that the resulting embedding space seems to lack some of the structure of its purely text-based alternatives. For instance, while text embeddings have been long noted to satisfy analogies in embedding space using vector arithmetic, CLIP has no such property. In this paper, we propose an approach to natively train CLIP in a contrastive manner to reason about differences in embedding space. We finetune CLIP so that text descriptions of differences between images correspond to their difference in image embedding space, using synthetically generated data with large language models on image-caption paired datasets. We first demonstrate that our approach yields significantly improved capabilities in ranking images by a certain attribute (e.g., elephants are larger than cats), which is useful in retrieval or constructing attribute-based classifiers, and improved zeroshot classification performance on many downstream image classification tasks. In addition, our approach enables a new mechanism for inference that we refer to as comparative prompting, where we leverage prior knowledge of text descriptions of differences between classes of interest, achieving even larger performance gains in classification. Finally, we illustrate that the resulting embeddings obey a larger degree of geometric properties in embedding space, such as in text-to-image generation.
URL: https://openreview.net/forum?id=USNJFZTWPn
---
Title: UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting
Authors: Juncheng Liu, Chenghao Liu, Gerald Woo, Yiwei Wang, Bryan Hooi, Caiming Xiong, Doyen Sahoo
Abstract: Transformer-based models have emerged as powerful tools for multivariate time series forecasting (MTSF). However, existing Transformer models often fall short of capturing both intricate dependencies across variate and temporal dimensions in MTS data. Some recent models are proposed to separately capture variate and temporal dependencies through either two sequential or parallel attention mechanisms. However, these methods cannot directly and explicitly learn the intricate inter-series and intra-series dependencies. In this work, we first demonstrate that these dependencies are very important as they usually exist in real-world data. To directly model these dependencies, we propose a transformer-based model UniTST containing a unified attention mechanism on the flattened patch tokens. Additionally, we add a dispatcher module which reduces the complexity and makes the model feasible for a potentially large number of variates. Although our proposed model employs a simple architecture, it offers compelling performance as shown in our extensive experiments on several datasets for time series forecasting.
URL: https://openreview.net/forum?id=p3y5q4cvzV
---
Title: Class-wise Generalization Error: an Information-Theoretic analysis
Authors: Firas Laakom, Moncef Gabbouj, Jürgen Schmidhuber, Yuheng Bu
Abstract: Existing generalization theories for supervised learning typically take a holistic approach and provide bounds for the expected generalization over the whole data distribution, which implicitly assumes that the model generalizes similarly for all different classes. In practice, however, there are significant variations in generalization performance among different classes, which cannot be captured by the existing generalization bounds. In this work, we tackle this problem by theoretically studying the class-generalization error, which quantifies the generalization performance of the model for each individual class. We derive a novel information-theoretic bound for class-generalization error using the KL divergence, and we further obtain several tighter bounds using recent advances in conditional mutual information
bound, which enables practical evaluation. We empirically validate our proposed bounds in various neural networks and show that they accurately capture the complex class-generalization behavior. Moreover, we demonstrate that the theoretical tools developed in
this work can be applied in several other applications.
URL: https://openreview.net/forum?id=asW4VcDFpi
---
Title: A Reproducibility Study of Decoupling Feature Extraction and Classification Layers for Calibrated Neural Networks
Authors: Johanna D'Ciofalo Khodaverdian, Eric Banzuzi, Katharina Deckenbach
Abstract: Many neural networks, especially over-parameterized ones, suffer from poor calibration and overconfidence. To address this, Jordahn & Olmos (2024) recently proposed a Two-Stage Training (TST) procedure that decouples the training of feature extraction and classification layers. In this study, we replicate their findings and extend their work through a series of ablation studies. We reproduce their main results and find that most of them replicate, with slight deviation for CIFAR100. Additionally, we extend the author's results by exploring the impact of different model architectures, Monte Carlo (MC) sample sizes, and classification head designs. We further compare the method with focal loss -- an implicit regularization technique known to improve calibration -- and investigate whether calibration can be improved further by combining the two methods. Beyond focal loss, we also evaluate the effect of incorporating other similar regularization techniques such as label smoothing and L2 regularization during two-stage training. We find that calibration can be improved even further by using focal loss in the first training stage of two-stage training. Similar improvements are observed when combining two-stage training with label smoothing and L2 regularization. Our experiments validate the claims made by Jordahn & Olmos (2024), and show the transferability of the two-stage training to different architectures.
URL: https://openreview.net/forum?id=5Hwzd48ILf
---
Title: Qualifying Knowledge and Knowledge Sharing in Multilingual Models
Authors: Nicolas Guerin, Ryan M. Nefdt, Emmanuel Chemla
Abstract: Pre-trained language models (PLMs) have demonstrated a remarkable ability to encode factual knowledge. However, the mechanisms underlying how this knowledge is stored and retrieved remain poorly understood, with important implications for AI interpretability and safety. In this paper, we disentangle the multifaceted nature of knowledge: successfully completing a knowledge retrieval task (e.g., “{The capital of France is __”) involves mastering underlying concepts (e.g., France, Paris), relationships between these concepts (e.g., capital of) and the structure of prompts, including the language of the query. We propose to disentangle these distinct aspects of knowledge and apply this typology to offer a critical view of neuron-level knowledge attribution techniques. For concreteness, we focus on Dai et al.'s (2022) Knowledge Neurons (KNs) across multiple PLMs (BERT, OPT, Llama and Gemma), testing 10 natural languages and additional unnatural languages (e.g. Autoprompt).
Our key contributions are twofold: (i) we show that KNs come in different flavors, some indeed encoding entity level concepts, some having a much less transparent, more polysemantic role , and (ii) we address the problem of cross-linguistic knowledge sharing at the neuron level, more specifically we uncover an unprecedented overlap in KNs across up to all of the 10 languages we tested, pointing to the existence of a partially unified, language-agnostic retrieval system. To do so, we introduce and release the Multi-ParaRel dataset, an extension of ParaRel, featuring prompts and paraphrases for cloze-style knowledge retrieval tasks in parallel over 10 languages.
URL: https://openreview.net/forum?id=hnpB3SRbZj
---
Title: Walking on the Fiber: A Simple Geometric Approximation for Bayesian Neural Networks
Authors: Alfredo Reichlin, Miguel Vasco, Danica Kragic
Abstract: Bayesian Neural Networks provide a principled framework for uncertainty quantification by modeling the posterior distribution of network parameters. However, exact posterior inference is computationally intractable, and widely used approximations like the Laplace method struggle with scalability and posterior accuracy in modern deep networks. In this work, we revisit sampling techniques for posterior exploration, proposing a simple variation tailored to efficiently sample from the posterior in over-parameterized networks by leveraging the low-dimensional structure of loss minima. Building on this, we introduce a model that learns a deformation of the parameter space, enabling rapid posterior sampling without requiring iterative methods. Empirical results demonstrate that our approach achieves competitive posterior approximations with improved scalability compared to recent refinement techniques. These contributions provide a practical alternative for Bayesian inference in deep learning.
URL: https://openreview.net/forum?id=NsuPykrjOd
---
Title: Successor Clusters: A Behavior Basis for Unsupervised Zero-Shot Reinforcement Learning
Authors: Louis Bagot, Lucas Nunes Alegre, Steven Latre, Kevin Mets, Bruno Castro da Silva
Abstract: In this work, we introduce Successor Clusters (SCs), a novel method for tackling unsupervised zero-shot reinforcement learning (RL) problems. The goal in this setting is to directly identify policies capable of optimizing any given reward functions without requiring further learning after an initial reward-free training phase. Existing state-of-the-art techniques leverage Successor Features (SFs)---functions capable of characterizing a policy's expected discounted sum of a set of $d$ reward features. Importantly, however, the performance of existing techniques depends critically on how well the reward features enable arbitrary reward functions of interest to be linearly approximated. We introduce a novel and principled approach for constructing reward features and prove that they allow for any Lipschitz reward functions to be approximated arbitrarily well. Furthermore, we mathematically derive upper bounds on the corresponding approximation errors. Our method constructs features by clustering the state space via a novel distance metric quantifying the minimal expected number of timesteps needed to transition between any state pairs. Building on these theoretical contributions, we introduce Successor Clusters (SCs), a variant of the successor features framework capable of predicting the time spent by a policy in different regions of the state space. We demonstrate that, after a pre-training phase, our method can approximate and maximize any new reward functions in a zero-shot manner. Importantly, we also formally show that as the number and quality of clusters increase, the set of policies induced by Successor Clusters converges to a set containing the optimal policy for any new task. Moreover, we show that our technique naturally produces interpretable features, enabling applications such as visualizing the sequence of state regions an agent is likely to visit while solving a task. Finally, we empirically demonstrate that our method outperforms state-of-the-art SF-based competitors in challenging continuous control benchmarks, achieving superior zero-shot performance and lower reward approximation error.
URL: https://openreview.net/forum?id=UB22Tt3sfF
---
Title: Setting the Record Straight on Transformer Oversmoothing
Authors: Gbetondji Jean-Sebastien Dovonon, Michael M. Bronstein, Matt Kusner
Abstract: Transformer-based models have recently become wildly successful across a diverse set of domains. At the same time, recent work has shown empirically and theoretically that Transformers are inherently limited. Specifically, they argue that as model depth increases, Transformers oversmooth, i.e., inputs become more and more similar. A natural question is: How can Transformers achieve these successes given this shortcoming? In this work we test these observations empirically and theoretically and uncover a number of surprising findings. We find that there are cases where feature similarity increases but, contrary to prior results, this is not inevitable, even for existing pre-trained models. Theoretically, we show that smoothing behavior depends on the eigenspectrum of the value and projection weights. We verify this empirically and observe that the sign of layer normalization weights can influence this effect. Our analysis reveals a simple way to parameterize the weights of the Transformer update equations to influence smoothing behavior. We hope that our findings give ML researchers and practitioners additional insight into how to develop future Transformer-based models.
URL: https://openreview.net/forum?id=HHI6qWLFF1
---
Title: No $D_{train}$: Model-Agnostic Counterfactual Explanations Using Reinforcement Learning
Authors: Xiangyu Sun, Raquel Aoki, Kevin H. Wilson
Abstract: Machine learning (ML) methods have experienced significant growth in the past decade, yet their practical application in high-impact real-world domains has been hindered by their opacity. When ML methods are responsible for making critical decisions, stakeholders often require insights into how to alter these decisions. Counterfactual explanations (CFEs) have emerged as a solution, offering interpretations of opaque ML models and providing a pathway to transition from one decision to another. However, most existing CFE methods require access to the model's training dataset, few methods can handle multivariate time-series, and none of model-agnostic CFE methods can handle multivariate time-series without training datasets. These limitations can be formidable in many scenarios. In this paper, we present NTD-CFE, a novel model-agnostic CFE method based on reinforcement learning (RL) that generates CFEs when training datasets are unavailable. NTD-CFE is suitable for both static and multivariate time-series datasets with continuous and discrete features. NTD-CFE reduces the CFE search space from a multivariate time-series domain to a lower dimensional space and addresses the problem using RL. Users have the flexibility to specify non-actionable, immutable, and preferred features, as well as causal constraints. We demonstrate the performance of NTD-CFE against four baselines on several datasets and find that, despite not having access to a training dataset, NTD-CFE finds CFEs that make significantly fewer and significantly smaller changes to the input time-series. These properties make CFEs more actionable, as the magnitude of change required to alter an outcome is vastly reduced. The code is available in the supplementary material.
URL: https://openreview.net/forum?id=egNzAG9rOu
---
Title: Task Arithmetic Through The Lens Of One-Shot Federated Learning
Authors: Zhixu Tao, Ian Mason, Sanjeev Kulkarni, Xavier Boix
Abstract: Task Arithmetic is a model merging technique that enables the combination of multiple models' capabilities into a single model through simple arithmetic in the weight space, without the need for additional fine-tuning or access to the original training data. However, the factors that determine the success of Task Arithmetic remain unclear. In this paper, we examine Task Arithmetic for multi-task learning by framing it as a one-shot Federated Learning problem. We demonstrate that Task Arithmetic is mathematically equivalent to the commonly used algorithm in Federated Learning, called Federated Averaging (FedAvg). By leveraging well-established theoretical results from FedAvg, we identify two key factors that impact the performance of Task Arithmetic: data heterogeneity and training heterogeneity. To mitigate these challenges, we adapt several algorithms from Federated Learning to improve the effectiveness of Task Arithmetic. Our experiments demonstrate that applying these algorithms can often significantly boost performance of the merged model compared to the original Task Arithmetic approach. This work bridges Task Arithmetic and Federated Learning, offering new theoretical perspectives on Task Arithmetic and improved practical methodologies for model merging.
URL: https://openreview.net/forum?id=Cgyo7S7Oy0
---
Title: Low-rank Momentum Factorization for Memory Efficient Training
Authors: Pouria Mahdavinia, Mehrdad Mahdavi
Abstract: Fine-tuning large foundation models presents significant memory challenges due to stateful optimizers like AdamW, often requiring several times more GPU memory than inference. While memory-efficient methods like parameter-efficient fine-tuning (e.g., LoRA) and optimizer state compression exist, recent approaches like GaLore bridge these by using low-rank gradient projections and subspace moment accumulation. However, such methods may struggle with fixed subspaces or computationally costly offline resampling (e.g., requiring full-matrix SVDs). We propose Momentum Factorized SGD (MoFaSGD), which maintains a dynamically updated low-rank SVD representation of the first-order momentum, closely approximating its full-rank counterpart throughout training. This factorization enables a memory-efficient fine-tuning method that adaptively updates the optimization subspace at each iteration. Crucially, MoFaSGD leverages the computed low-rank momentum factors to perform efficient spectrally normalized updates, offering an alternative to subspace moment accumulation. We establish theoretical convergence guarantees for MoFaSGD, proving it achieves an optimal rate for non-convex stochastic optimization under standard assumptions. Empirically, we demonstrate MoFaSGD's effectiveness on large language model alignment benchmarks, achieving a competitive trade-off between memory reduction (comparable to LoRA) and performance compared to state-of-the-art low-rank optimization methods. Our implementation is available at \url{https://github.com/pmahdavi/MoFaSGD}.
URL: https://openreview.net/forum?id=W3D3TVo9a3
---
Title: A Comprehensive Survey of Contamination Detection Methods in Large Language Models
Authors: Mathieu Ravaut, Bosheng Ding, Fangkai Jiao, Hailin Chen, Xingxuan Li, Ruochen Zhao, Chengwei Qin, Caiming Xiong, Shafiq Joty
Abstract: With the rise of Large Language Models (LLMs) in recent years, abundant new opportunities are emerging, but also new challenges, among which contamination is quickly becoming critical. Business applications and fundraising in Artificial Intelligence (AI) have reached a scale at which a few percentage points gained on popular question-answering benchmarks could translate into dozens of millions of dollars, placing high pressure on model integrity. At the same time, it is becoming harder and harder to keep track of the data that LLMs have seen; if not impossible with closed-source models like GPT-4 and Claude-3 not divulging any information on the training set. As a result, contamination becomes a major issue: LLMs’ performance may not be reliable anymore, as the high performance may be at least partly due to their previous exposure to the data. This limitation jeopardizes real capability improvement in the field of NLP, yet, there remains a lack of methods on how to efficiently detect contamination. In this paper, we survey all recent work on contamination detection with LLMs, analyzing their methodologies and use cases to shed light on the appropriate usage of contamination detection methods. Our work calls the NLP research community’s attention into systematically taking into account contamination bias in LLM evaluation.
URL: https://openreview.net/forum?id=SxNMjbtdFm
---
Title: GenCAD: Image-Conditioned Computer-Aided Design Generation with Transformer-Based Contrastive Representation and Diffusion Priors
Authors: Md Ferdous Alam, Faez Ahmed
Abstract: The creation of manufacturable and editable 3D shapes through Computer-Aided Design (CAD) remains a highly manual and time-consuming task, hampered by the complex topology of boundary representations of 3D solids and unintuitive design tools. While most work in the 3D shape generation literature focuses on representations like meshes, voxels, or point clouds, practical engineering applications demand the modifiability and manufacturability of CAD models and the ability for multi-modal conditional CAD model generation. This paper introduces GenCAD, a generative model that employs autoregressive transformers with a contrastive learning framework and latent diffusion models to transform image inputs into parametric CAD command sequences, resulting in editable 3D shape representations. Extensive evaluations demonstrate that GenCAD significantly outperforms existing state-of-the-art methods in terms of the unconditional and conditional generations of CAD models. Additionally, the contrastive learning framework of GenCAD facilitates the retrieval of CAD models using image queries from large CAD databases, which is a critical challenge within the CAD community. Our results provide a significant step forward in highlighting the potential of generative models to expedite the entire design-to-production pipeline and
seamlessly integrate different design modalities.
URL: https://openreview.net/forum?id=e817c1wEZ6
---
Title: Model-free reinforcement learning with noisy actions for automated experimental control in optics
Authors: Lea Richtmann, Viktoria-S. Schmiesing, Dennis Wilken, Jan Heine, Aaron D Tranter, Avishek Anand, Tobias J. Osborne, Michèle Heurs
Abstract: Setting up and controlling optical systems is often a challenging and tedious task. The high number of degrees of freedom to control mirrors, lenses, or phases of light makes automatic control challenging, especially when the complexity of the system cannot be adequately modeled due to noise or non-linearities. Here, we show that reinforcement learning (RL) can overcome these challenges when coupling laser light into an optical fiber, using a model-free RL approach that trains directly on the experiment without pre-training on simulations. By utilizing the sample-efficient algorithms Soft Actor-Critic (SAC), Truncated Quantile Critics (TQC), or CrossQ, our agents learn to couple with 90% efficiency. A human expert reaches this efficiency, but the RL agents are quicker. In particular, the CrossQ agent outperforms the other agents in coupling speed while requiring only half the training time.
We demonstrate that direct training on an experiment can replace extensive system modeling. Our result exemplifies RL's potential to tackle problems in optics, paving the way for more complex applications where full noise modeling is not feasible.
URL: https://openreview.net/forum?id=DAYsM4zDNg
---
Title: Celo: Training Versatile Learned Optimizers on a Compute Diet
Authors: Abhinav Moudgil, Boris Knyazev, Guillaume Lajoie, Eugene Belilovsky
Abstract: Learned optimization has emerged as a promising alternative to hand-crafted optimizers, with the potential to discover stronger learned update rules that enable faster, hyperparameter-free training of neural networks. A critical element for practically useful learned optimizers, that can be used off-the-shelf after meta-training, is strong meta-generalization: the ability to apply the optimizers to new tasks. Recent state-of-the-art work in learned optimizers, VeLO (Metz et al., 2022), requires a large number of highly diverse meta-training tasks along with massive computational resources, 4000 TPU months, to achieve meta-generalization. This makes further improvements to such learned optimizers impractical. In this work, we identify several key elements in learned optimizer architectures and meta-training procedures that can lead to strong meta-generalization. We also propose evaluation metrics to reliably assess quantitative performance of an optimizer at scale on a set of evaluation tasks. Our proposed approach, Celo, makes a significant leap in improving the meta-generalization performance of learned optimizers and also outperforms tuned state-of-the-art optimizers on a diverse set of out-of-distribution tasks, despite being meta-trained for just 24 GPU hours.
URL: https://openreview.net/forum?id=SLqJbt4emY
---
Title: Gaussian Processes with Bayesian Inference of Covariate Couplings
Authors: Mattia Rosso, Juho Ylä-Jääski, Zheyang Shen, Markus Heinonen, Maurizio Filippone
Abstract: Gaussian processes are powerful probabilistic models that are often coupled with ARD capable of uncovering the importance of individual covariates. We develop covariances characterized by affine transformations of the inputs, formalized via a precision matrix between covariates, which can uncover covariate couplings for enhanced interpretability. We study a range of couplings priors from Wishart to Horseshoe and present fully Bayesian inference of such precision matrices within sparse Gaussian processes. We empirically demonstrate the efficacy and interpretability of this approach.
URL: https://openreview.net/forum?id=fameEAljo3
---
Title: MAMUT: A Novel Framework for Modifying Mathematical Formulas for the Generation of Specialized Datasets for Language Model Training
Authors: Jonathan Drechsel, Anja Reusch, Steffen Herbold
Abstract: Mathematical formulas are a fundamental and widely used component in various scientific fields, serving as a universal language for expressing complex concepts and relationships. While state-of-the-art transformer models excel in processing and understanding natural language, they encounter challenges with mathematical notation, which involves a complex structure and diverse representations. This study focuses on the development of specialized training datasets to enhance the encoding of mathematical content. We introduce Math Mutator (MAMUT), a framework capable of generating equivalent and falsified versions of a given mathematical formula in LaTeX notation, effectively capturing the mathematical variety in notation of the same concept. Based on MAMUT, we have generated four large mathematical datasets containing diverse notation, which can be used to train language models with enhanced mathematical embeddings. Experiments show that models trained on these datasets exhibit new SoTA performance on mathematical retrieval tasks. We publish our code, generated datasets, and pretrained mathematical models: https://github.com/aieng-lab/math-mutator.
URL: https://openreview.net/forum?id=khODmRpQEx
---
Title: A Baseline Method for Removing Invisible Image Watermarks using Deep Image Prior
Authors: Hengyue Liang, Taihui Li, Ju Sun
Abstract: Image watermarks have been considered a promising technique to help detect AI-generated content, which can be used to protect copyright or prevent fake image abuse. In this work, we present a black-box method for removing invisible image watermarks, without the need of any dataset of watermarked images or any knowledge about the watermark system. Our approach is simple to implement: given a single watermarked image, we regress it by deep image prior (DIP). We show that from the intermediate steps of DIP one can reliably find an evasion image that can remove invisible watermarks while preserving high image quality. Due to its unique working mechanism and practical effectiveness, we advocate including DIP as a baseline invasion method for benchmarking the robustness of watermarking systems. Finally, by showing the limited ability of DIP and other existing black-box methods in evading training-based visible watermarks, we discuss the positive implications on the practical use of training-based visible watermarks to prevent misinformation abuse.
URL: https://openreview.net/forum?id=g85Vxlrq0O
---
Title: Learning Federated Neural Graph Databases for Answering Complex Queries from Distributed Knowledge Graphs
Authors: Qi Hu, Weifeng Jiang, Haoran Li, Zihao Wang, Jiaxin Bai, Qianren Mao, Yangqiu Song, Lixin Fan, Jianxin Li
Abstract: The increasing demand for deep learning-based foundation models has highlighted the importance of efficient data retrieval mechanisms. Neural graph databases (NGDBs) offer a compelling solution, leveraging neural spaces to store and query graph-structured data, thereby enabling LLMs to access precise and contextually relevant information. However, current NGDBs are constrained to single-graph operation, limiting their capacity to reason across multiple, distributed graphs. Furthermore, the lack of support for multi-source graph data in existing NGDBs hinders their ability to capture the complexity and diversity of real-world data. In many applications, data is distributed across multiple sources, and the ability to reason across these sources is crucial for making informed decisions. This limitation is particularly problematic when dealing with sensitive graph data, as directly sharing and aggregating such data poses significant privacy risks. As a result, many applications that rely on NGDBs are forced to choose between compromising data privacy or sacrificing the ability to reason across multiple graphs. To address these limitations, we propose to learn Federated Neural Graph DataBase (FedNGDB), a pioneering systematic framework that empowers privacy-preserving reasoning over multi-source graph data. FedNGDB leverages federated learning to collaboratively learn graph representations across multiple sources, enriching relationships between entities, and improving the overall quality of graph data. Unlike existing methods, FedNGDB can handle complex graph structures and relationships, making it suitable for various downstream tasks. We evaluate FedNGDBs on three real-world datasets, demonstrating its effectiveness in retrieving relevant information from multi-source graph data while keeping sensitive information secure on local devices. Our results show that FedNGDBs can efficiently retrieve answers to cross-graph queries, making it a promising approach for LLMs and other applications that rely on efficient data retrieval mechanisms.
URL: https://openreview.net/forum?id=3K1LRetR6Y
---
Title: Bayesian Neighborhood Adaptation for Graph Neural Networks
Authors: Paribesh Regmi, Rui Li, Kishan K C
Abstract: The neighborhood scope (i.e., number of hops) where graph neural networks (GNNs) aggregate information to characterize a node's statistical property is critical to GNNs' performance. Two-stage approaches, training and validating GNNs for every pre-specified neighborhood scope to search for the best setting, is a time-consuming task and tends to be biased due to the search space design. How to adaptively determine proper neighborhood scopes for the aggregation process for both homophilic and heterophilic graphs remains largely unexplored. We thus propose to model the GNNs' message-passing behavior on a graph as a stochastic process by treating the number of hops as a beta process. This Bayesian framework allows us to infer the most plausible neighborhood scope for message aggregation simultaneously with the optimization of GNN parameters. Our theoretical analysis shows that the scope inference improves the expressivity of a GNN. Experiments on benchmark homophilic and heterophilic datasets show that the proposed method is compatible with state-of-the-art GNN variants, achieving competitive or superior performance on the node classification task, and providing well-calibrated predictions.
URL: https://openreview.net/forum?id=2zEemRib3a
---
Title: Approximations to worst-case data dropping: unmasking failure modes
Authors: Jenny Y. Huang, David R. Burt, Yunyi Shen, Tin D. Nguyen, Tamara Broderick
Abstract: A data analyst might worry about generalization if dropping a very small fraction of data points from a study could change its substantive conclusions. Checking this non-robustness directly poses a combinatorial optimization problem and is intractable even for simple models and moderate data sizes. Recently various authors have proposed a diverse set of approximations to detect this non-robustness. In the present work, we show that, even in a setting as simple as ordinary least squares (OLS) linear regression, many of these approximations can fail to detect (true) non-robustness in realistic data arrangements. We focus on OLS in the present work due its widespread use and since some approximations work only for OLS. Across our synthetic and real-world data sets, we find that a simple recursive greedy algorithm is the sole algorithm that does not fail any of our tests and also that it can be orders of magnitude faster to run than some competitors.
URL: https://openreview.net/forum?id=m6EQ6YdPXV
---
Title: Large Language Model Confidence Estimation via Black-Box Access
Authors: Tejaswini Pedapati, Amit Dhurandhar, Soumya Ghosh, Soham Dan, Prasanna Sattigeri
Abstract: Estimating uncertainty or confidence in the responses of a model can be significant in evaluating trust not only in the responses, but also in the model as a whole. In this paper, we explore the problem of estimating confidence for responses of large language models
(LLMs) with simply black-box or query access to them. We propose a simple and extensible framework where, we engineer novel features and train a (interpretable) model (viz. logistic regression) on these features to estimate the confidence. We empirically demonstrate that
our simple framework is effective in estimating confidence of Flan-ul2, Llama-13b, Mistral-7b and GPT-4 on four benchmark Q&A tasks as well as of Pegasus-large and BART-large on two benchmark summarization tasks with it surpassing baselines by even over 10% (on AU-ROC) in some cases. Additionally, our interpretable approach provides insight into features that are predictive of confidence, leading to the interesting and useful discovery that our confidence models built for one LLM generalize zero-shot across others on a given dataset.
URL: https://openreview.net/forum?id=WrWYChkyRI
---
Title: Combinatorial Multi-armed Bandits: Arm Selection via Group Testing
Authors: Arpan Mukherjee, Shashanka Ubaru, Keerthiram Murugesan, Karthikeyan Shanmugam, Ali Tajer
Abstract: This paper considers the problem of combinatorial multi-armed bandits with semi-bandit feedback and a cardinality constraint on the super-arm size. Existing algorithms for solving this problem typically involve two key sub-routines: (1) a *parameter estimation* routine that sequentially estimates a set of base-arm parameters, and (2) a *super-arm selection* policy for selecting a subset of base arms deemed optimal based on these parameters. State-of-the-art algorithms assume access to an *exact* oracle for super-arm selection with unbounded computational power. At each instance, this oracle evaluates a list of score functions, the number of which grows as low as linearly and as high as exponentially with the number of arms. This can be prohibitive in the regime of a large number of arms. This paper introduces a novel realistic alternative to the perfect oracle. This algorithm uses a combination of *group-testing* for selecting the super arms and *quantized* Thompson sampling for parameter estimation. Under a general separability assumption on the reward function, the proposed algorithm reduces the complexity of the super-arm-selection oracle to be *logarithmic* in the number of base arms while achieving the same regret order as the state-of-the-art algorithms that use exact oracles. This translates to *at least an exponential* reduction in complexity compared to the oracle-based approaches.
URL: https://openreview.net/forum?id=Mq59rTnIfE
---
Title: Where to Intervene: Action Selection in Deep Reinforcement Learning
Authors: Wenbo Zhang, Hengrui Cai
Abstract: Deep reinforcement learning (RL) has gained widespread adoption in recent years but faces significant challenges, particularly in unknown and complex environments. Among these, high-dimensional action selection stands out as a critical problem. Existing works often require a sophisticated prior design to eliminate redundancy in the action space, relying heavily on domain expert experience or involving high computational complexity, which limits their generalizability across different RL tasks. In this paper, we address these challenges by proposing a general data-driven action selection approach with model-free and computationally friendly properties. Our method not only selects minimal sufficient actions but also controls the false discovery rate via knockoff sampling. More importantly, we seamlessly integrate the action selection into deep RL methods during online training. Empirical experiments validate the established theoretical guarantees, demonstrating that our method surpasses various alternative techniques in terms of both performance in variable selection and overall achieved rewards.
URL: https://openreview.net/forum?id=D3au9XkWuy
---
Title: Counting Hours, Counting Losses: The Toll of Unpredictable Work Schedules on Financial Security
Authors: Pegah Nokhiz, Aravinda Kanchana Ruwanpathirana, Aditya Bhaskara, Suresh Venkatasubramanian
Abstract: Financial instability is a pressing concern in the United States, with drivers that include growing employment disparities and insufficient wages. While research typically focuses on financial aspects such as income inequality in precarious work environments, there is a tendency to overlook the time-related aspect of unstable work schedules. The inability to rely on a consistent work schedule not only leads to burnout and conflicts between work and family life but also results in financial shocks that directly impact workers' income and assets. Unforeseen fluctuations in earnings pose challenges in financial planning, affecting decisions regarding savings and spending, and ultimately undermining individuals' long-term financial stability and well-being.
Our objective in this study is to understand how unforeseen fluctuations in earnings exacerbate financial fragility by investigating the extent to which individuals' financial management depends on their ability to anticipate and plan for future events. To answer this question, we present a computational framework to model real-time consumption decisions under income uncertainty, drawing on advances in online planning and reinforcement learning (RL) with lookahead. We introduce a novel online algorithm that enables utility-maximizing agents to dynamically adapt consumption choices in response to financial shocks, leveraging partial deterministic information about future income. This approach forms the basis of our simulation framework, which models how workers manage consumption in the face of variable work schedules and the imperative to avoid financial ruin.
Through theoretical analysis, we quantify the utility advantage conferred by varying levels of lookahead. Empirical simulations demonstrate how increased lookahead improves financial utility. That is, with this framework, we demonstrate both theoretically and empirically how a worker's capacity to anticipate schedule changes enhances their long-term utility. Conversely, the inability to predict future events can worsen workers' financial instability. Moreover, our framework enables us to explore policy interventions aimed at mitigating the problem of schedule uncertainty. By modeling both individual behavior and potential policy interventions (e.g., advance scheduling regulations), our framework draws on ideas from machine learning and reinforcement learning to inform economic questions surrounding information access in financial planning.
URL: https://openreview.net/forum?id=PEZz2i9kiP
---
Title: Does confidence calibration improve conformal prediction?
Authors: HuaJun Xi, Jianguo Huang, Kangdao Liu, Lei Feng, Hongxin Wei
Abstract: Conformal prediction is an emerging technique for uncertainty quantification that constructs prediction sets guaranteed to contain the true label with a predefined probability. Previous works often employ temperature scaling to calibrate classifiers, assuming that confidence calibration benefits conformal prediction. However, the specific impact of confidence calibration on conformal prediction remains underexplored. In this work, we make two key discoveries about the impact of confidence calibration methods on adaptive conformal prediction. Firstly, we empirically show that current confidence calibration methods (e.g., temperature scaling) typically lead to larger prediction sets in adaptive conformal prediction. Secondly, by investigating the role of temperature value, we observe that high-confidence predictions can enhance the efficiency of adaptive conformal prediction. Theoretically, we prove that predictions with higher confidence result in smaller prediction sets on expectation. This finding implies that the rescaling parameters in these calibration methods, when optimized with cross-entropy loss, might counteract the goal of generating efficient prediction sets. To address this issue, we propose \textbf{Conformal Temperature Scaling} (ConfTS), a variant of temperature scaling with a novel loss function designed to enhance the efficiency of prediction sets. This approach can be extended to optimize the parameters of other post-hoc methods of confidence calibration. Extensive experiments demonstrate that our method improves existing adaptive conformal prediction methods in both image and text classification tasks.
URL: https://openreview.net/forum?id=6DDaTwTvdE
---
Title: Cluster and Predict Latents Patches for Improved Masked Image Modeling
Authors: Timothée Darcet, Federico Baldassarre, Maxime Oquab, Julien Mairal, Piotr Bojanowski
Abstract: Masked Image Modeling (MIM) offers a promising approach to self-supervised representation learning, however existing MIM models still lag behind the state-of-the-art.
In this paper, we systematically analyze target representations, loss functions, and architectures, to introduce CAPI -- a novel pure-MIM framework that relies on the prediction of latent clusterings.
Our approach leverages a clustering-based loss, which is stable to train, and exhibits promising scaling properties.
Our ViT-L backbone, CAPI, achieves 83.8\% accuracy on ImageNet and 32.1\% mIoU on ADE20K with simple linear probes, substantially outperforming previous MIM methods and approaching the performance of the current state-of-the-art, DINOv2.
URL: https://openreview.net/forum?id=Ycmz7qJxUQ
---
Title: Fully Automatic Neural Network Reduction for Formal Verification
Authors: Tobias Ladner, Matthias Althoff
Abstract: Formal verification of neural networks is essential before their deployment in safety-critical applications.
However, existing methods for formally verifying neural networks are not yet scalable enough to handle practical problems under strict time constraints.
We address this challenge by introducing a fully automatic and sound reduction of neural networks using reachability analysis.
The soundness ensures that the verification of the reduced network entails the verification of the original network.
Our sound reduction approach is applicable to neural networks with any type of element-wise activation function, such as ReLU, sigmoid, and tanh.
The network reduction is computed on the fly while simultaneously verifying the original network and its specification.
All parameters are automatically tuned to minimize the network size without compromising verifiability.
We further show the applicability of our approach to convolutional neural networks by explicitly exploiting similar neighboring pixels.
Our evaluation shows that our approach reduces large neural networks to a fraction of the original number of neurons
and thus shortens the verification time to a similar degree.
URL: https://openreview.net/forum?id=gmflcWlVMl
---
New submissions
===============
Title: Minimax Multi-Target Conformal Prediction with Applications to Imaging Inverse Problems
Abstract: In ill-posed imaging inverse problems, uncertainty quantification remains a fundamental challenge, especially in safety-critical applications. Recently, conformal prediction has been used to quantify the uncertainty that the inverse problem contributes to downstream tasks like image classification, image quality assessment, fat mass quantification, etc. While existing works handle only a scalar estimation target, practical applications often involve multiple targets. In response, we propose a minimax approach to multi-target conformal prediction that provides tight prediction intervals while ensuring joint marginal coverage. We then outline how our minimax approach can be applied to multi-metric blind image quality assessment, multi-task uncertainty quantification, and multi-round measurement acquisition. Finally, we numerically demonstrate the benefits of our minimax method, relative to existing multi-target conformal prediction methods, using magnetic resonance imaging (MRI) data.
URL: https://openreview.net/forum?id=53FEYwDQK0
---
Title: Neural Spatiotemporal Point Processes: Trends and Challenges
Abstract: Spatiotemporal point processes (STPPs) are probabilistic models for events occurring in continuous space and time. Real-world event data often exhibits intricate dependencies and heterogeneous dynamics. By incorporating modern deep learning techniques, STPPs can model these complexities more effectively than traditional approaches. Consequently, the fusion of neural methods with STPPs has become an active and rapidly evolving research area. In this review, we categorise existing approaches, unify key design choices, and explain the challenges of working with this data modality. We further highlight emerging trends and diverse application domains. Finally, we identify open challenges and gaps in the literature.
URL: https://openreview.net/forum?id=N69lSYWkMw
---
Title: Inherently Faithful Attention Maps for Vision Transformers
Abstract: We introduce an attention-based method that uses learned binary attention masks to ensure that only attended image regions influence the prediction. Context can strongly affect object perception, sometimes leading to biased representations, particularly when objects appear in out-of-distribution backgrounds. At the same time, many image-level object-centric tasks require identifying relevant regions, often requiring context. To address this conundrum, we propose a two-stage framework: stage 1 processes the full image to discover object parts and identify task-relevant regions, while stage 2 leverages input attention masking to restrict its receptive field to these regions, enabling a focused analysis while filtering out potentially spurious information. Both stages are trained jointly, allowing stage 2 to refine stage 1. Extensive experiments across diverse benchmarks demonstrate that our approach significantly improves robustness against spurious correlations and out-of-distribution backgrounds.
URL: https://openreview.net/forum?id=EM91M3MUGI
---
Title: Length independent generalization bounds for deep SSM architectures via Rademacher contraction and stability constraints
Abstract: Deep SSM models like S4, S5, and LRU are made of sequential blocks that combine State-Space Model (SSM) layers with neural networks, achieving excellent performance on long-range sequences. In this paper we provide a PAC bound that holds for non-selective architectures with stable SSM blocks and does not depend on the length of the input sequence. Imposing stability of the SSM blocks is a standard practice in the literature, and it is known to help performance. Our results provide a theoretical justification for the use of stable SSM blocks as the proposed PAC bound decreases as the degree of stability of the SSM blocks increases.
URL: https://openreview.net/forum?id=Vo6wHBv07k
---
Title: Two-Step Offline Preference-Based Reinforcement Learning on Explicitly Constrained Policies
Abstract: Preference-based reinforcement learning (PBRL) in the offline setting has succeeded greatly in industrial applications such as chatbots. A two-step learning framework that learns a reward model from the dataset first and then optimizes a policy over the learned reward model through reinforcement learning has been widely adopted for the problem. However, such a method faces challenges from the risk of reward hacking and the complexity of reinforcement learning. To overcome the challenge, our insight is that both challenges come from the state-actions not supported in the dataset. Such state-actions are unreliable and increase the complexity of the reinforcement learning problem. Based on the insight, we develop a novel two-step learning method called PRC: preference-based reinforcement learning on explicitly constrained policies. The high-level idea is to limit the reinforcement learning agent to optimize over policies supported on an explicitly constrained action space that excludes the out-of-distribution state-actions. We empirically verify that our method has high learning efficiency on various datasets in robotic control environments.
URL: https://openreview.net/forum?id=LxPg5GJuY3
---
Title: Learning Representations for Independence Testing
Abstract: Many tools exist to detect dependence between random variables, a core question across a wide range of machine learning, statistical, and scientific endeavors. Although several statistical tests guarantee eventual detection of any dependence with enough samples, standard tests may require an exorbitant amount of samples for detecting subtle dependencies between high-dimensional random variables with complex distributions. In this work, we study two related ways to learn powerful independence tests. First, we show how to construct powerful statistical tests with finite-sample validity by using variational estimators of mutual information, such as the InfoNCE or NWJ estimators. Second, we establish a close connection between these variational mutual information-based tests and tests based on the Hilbert-Schmidt Independence Criterion (HSIC); in particular, learning a variational bound (typically parameterized by a deep network) for mutual information is closely related to learning a kernel for HSIC. Finally, we show how to, rather than selecting a representation to maximize the statistic itself, select a representation which can maximize the power of a test, in either setting; we term the former case a Neural Dependency Statistic (NDS). While HSIC power optimization has been recently considered in the literature, we correct some important misconceptions and expand to considering deep kernels. In our experiments, while all approaches can yield powerful tests with exact level control, optimized HSIC tests generally outperform the other approaches on difficult problems of detecting structured dependence.
URL: https://openreview.net/forum?id=pDvKoXRsnW
---
Title: Explaining How Quantization Disparately Skews a Model
Abstract: Post Training Quantization (PTQ) is widely adopted due to its high compression capacity and speed with minimal impact on accuracy. However, we observed that disparate impacts are exacerbated by quantization, especially for minority groups. Our analysis explains that
in the course of quantization there is a chain of factors attributed to a disparate impact across groups during forward and backward passes. We explore how the changes in weights and activations induced by quantization cause cascaded impacts in the network, resulting
in logits with lower variance, increased loss, and compromised group accuracies. We extend our study to verify the influence of these impacts on group gradient norms and eigenvalues of the Hessian matrix, providing insights into the state of the network from an optimization point of view. To mitigate these effects, we propose integrating mixed precision Quantization Aware Training (QAT) with dataset sampling methods and weighted loss functions, therefore providing fair deployment of quantized neural networks.
URL: https://openreview.net/forum?id=6ShdrLUNne
---
Title: Distributed Hierarchical Decomposition Framework for Multimodal Timeseries Prediction
Abstract: We consider a distributed time series forecasting problem where multiple distributed nodes each observing a local time series (of potentially different modality) collaborate to make both local and global forecasts. This problem is particularly challenging because each node only observes time series generated from a subset of sources, making it challenging to utilize correlations among different streams for accurate forecasting; and the data streams observed at each node may represent different modalities, leading to heterogeneous computational requirements among nodes. To tackle these challenges, we propose a hierarchical learning framework, consisting of multiple local models and a global model, and provide a suite of efficient training algorithms to achieve high local and global forecasting accuracy. We theoretically establish the convergence of the proposed framework and demonstrate the effectiveness of the proposed approach using several time series forecasting tasks, with the (somewhat surprising) observation that the proposed distributed models can match, or even outperform centralized ones.
URL: https://openreview.net/forum?id=fFKWs9HslJ
---
Title: Exponential tilting of subweibull distributions
Abstract: The class of subweibull distributions has recently been shown to generalize the important
properties of subexponential and subgaussian random variables. We describe alternative
characterizations of subweibull distributions and detail the conditions under which their tail
behavior is preserved after exponential tilting.
URL: https://openreview.net/forum?id=BQBk11IE7I
---
Title: Neural ODE and SDE Models for Adaptation and Planning in Model-Based Reinforcement Learning
Abstract: We investigate neural ordinary and stochastic differential equations (neural ODEs and SDEs) to model stochastic dynamics in fully and partially observed environments within a model-based reinforcement learning (RL) framework. Through a sequence of simulations, we show that neural SDEs more effectively capture transition dynamics’ inherent stochasticity, enabling high-performing policies with improved sample efficiency in challenging scenarios. We leverage neural ODEs and SDEs for efficient policy adaptation to changes in environment dynamics via inverse models, requiring only limited interactions with the new environment. To address partial observability, we introduce a latent SDE model that combines an ODE and a GAN-trained stochastic component in latent space. This model matches or exceeds the performance of the state-based SDE variant and outperforms ODE-based alternatives across stochastic variants of continuous control benchmarks, providing the first empirical demonstration of action-conditional latent neural SDEs for planning in such settings.
URL: https://openreview.net/forum?id=T6OrPlyPV4
---
Title: Koopman Embedded Equivariant Control
Abstract: An efficient way to control systems with unknown nonlinear dynamics is to find an appropriate embedding or representation for simplified approximation (e.g. linearization), which facilitates system identification and control synthesis. Nevertheless, there has been a lack of embedding methods that can guarantee (i) embedding the dynamical system comprehensively, including the vector fields (ODE form) of the dynamics, and (ii) preserving the consistency of control effect between the original and latent space. To address these challenges, we propose Koopman Embedded Equivariant Control (KEEC) to learn an embedding of the states and vector fields such that a Koopman operator is approximated as the latent dynamics. Due to the Koopman operator's linearity, learning the latent vector fields of the dynamics becomes simply solving linear equations. Thus in KEEC, the analytical form of the greedy control policy, which is dependent on the learned differential information of the dynamics and value function, is also simplified. Meanwhile, KEEC preserves the effectiveness of the control policy in the latent space by preserving the metric in two spaces. Our algorithm achieves superior performances in the experiments conducted on various control domains, including the image-based Pendulum, Lorenz-63 and the wave equation.
URL: https://openreview.net/forum?id=BO79sLEKce
---
Title: Subspace based Federated Unlearning
Abstract: Federated learning (FL) enables collaborative machine learning among multiple clients while preserving user data privacy by preventing the exchange of local data. However, when users request to leave the FL system, the trained FL model may still retain information about their contributions. To comply with the right to be forgotten, federated unlearning has been proposed, which aims to remove a designated client's influence from the FL model. Existing federated unlearning methods typically rely on storing historical parameter updates, which may be impractical in resource-constrained FL settings. In this paper, we propose a Subspace-based Federated Unlearning method (SFU) that addresses this challenge without requiring additional storage. SFU updates the model via gradient ascent constrained within a subspace, specifically the orthogonal complement of the gradient descent directions derived from the remaining clients. By projecting the ascending gradient of the target client onto this subspace, SFU can mitigate the contribution of the target client while maintaining model performance on the remaining clients. SFU is communication-efficient, requiring only one round of local training per client to transmit gradient information to the server for model updates. Extensive empirical evaluations on multiple datasets demonstrate that SFU achieves competitive unlearning performance while preserving model utility. Compared to representative baseline methods, SFU consistently shows promising results under various experimental settings.
URL: https://openreview.net/forum?id=KE2ZNl2lFP
---
Title: SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models
Abstract: This work explores two distinct approaches for enhancing reasoning abilities in Large Vision Language Models (LVLMs): supervised fine-tuning (SFT) and reinforcement learning (RL). To support the SFT approach, we curate a multimodal reasoning dataset with the complete reasoning trace guided by DeepSeek-R1. For the RL approach, we focus on GRPO and develop a training framework tailored to vision-language tasks with a composite reward system comprising four signals that address both visual perception and reasoning challenges. Our extensive experiments reveal that RL is a significantly more effective strategy than SFT for training reasoning VLMs. While SFT can assist models that initially struggle with following reasoning instructions, it often induces ``pseudo aha moments'' that degrade overall reasoning performance, implying that only a minimal amount of SFT data is necessary. In contrast, RL leads to substantial improvements, outperforming recent baseline models on a range of math reasoning tasks by at least 2% on average. We also present several intriguing findings --- \eg, combining SFT and GRPO also hurts the model performance, and stronger instruction-aligned LVLMs consistently lead to better results in RL. We hope these findings provide valuable insights into the development of reasoning-capable VLMs and guide future research in this area.
URL: https://openreview.net/forum?id=wZI5qkQeDF
---
Title: Byzantine-Robust Gossip: Insights from a Dual Approach
Abstract: Distributed learning has many computational benefits but is vulnerable to attacks from a subset of devices transmitting incorrect information. This paper investigates Byzantine-resilient algorithms in a decentralized setting, where devices communicate directly in a peer-to-peer manner within a communication network. We leverage the so-called dual approach for decentralized optimization and propose a Byzantine-robust algorithm. We provide convergence guarantees in the average consensus subcase, discuss the potential of the dual approach beyond this subcase, and re-interpret existing algorithms using the dual framework. Lastly, we experimentally show the soundness of our method.
URL: https://openreview.net/forum?id=wrLiUpfk4s
---
Title: Responsibility: A General Instance and Training Process-based Explainable AI Approach
Abstract: Explainable Artificial Intelligence (XAI) methods focus on helping human users better understand the decision making of an AI agent. However, many modern XAI approaches are not actionable to end users, particularly those without prior AI or ML knowledge. In this paper, we formally define and extend an XAI approach called Responsibility, which identifies the most responsible training instance for a particular model decision based on observing the model's training process. This instance can then be presented as an explanation: ``this is what the AI agent learned that led to that decision.'' We present experimental results across a number of domains and architectures, along with the results of a user study. Our results demonstrate that Responsibility can help improve the performance of both human end users and secondary ML models.
URL: https://openreview.net/forum?id=CqgblPNfcw
---
Title: CoCoIns: Consistent Subject Generation via Contrastive Instantiated Concepts
Abstract: While text-to-image generative models can synthesize diverse and faithful content, subject variation across multiple creations limits the application in long content generation. Existing approaches require time-consuming tuning, references for all subjects, or access to other creations. We introduce Contrastive Concept Instantiation (CoCoIns) to effectively synthesize consistent subjects across multiple independent creations. The framework consists of a generative model and a mapping network, which transforms input latent codes into pseudo-words associated with certain instances of concepts. Users can generate consistent subjects with the same latent codes. To construct such associations, we propose a contrastive learning approach that trains the network to differentiate the combination of prompts and latent codes. Extensive evaluations of human faces with a single subject show that CoCoIns performs comparably to existing methods while maintaining higher flexibility. We also demonstrate the potential of extending CoCoIns to multiple subjects and other object categories. The source code and models will be released.
URL: https://openreview.net/forum?id=fPZ7DNlOSn
---
Title: SynCo: Synthetic Hard Negatives for Contrastive Visual Representation Learning
Abstract: Contrastive learning has become a dominant approach in self-supervised visual representation learning, but efficiently leveraging hard negatives, which are samples closely resembling the anchor, remains challenging. We introduce SynCo (Synthetic negatives in Contrastive learning), a novel approach that improves model performance by generating synthetic hard negatives on the representation space. Building on the MoCo framework, SynCo introduces six strategies for creating diverse synthetic hard negatives "on-the-fly" with minimal computational overhead. SynCo achieves faster training and strong representation learning, surpassing MoCo-v2 by +0.4% and MoCHI by +1.0% on ImageNet ILSVRC-2012 linear evaluation. It also transfers more effectively to detection tasks achieving strong results on PASCAL VOC detection (57.2% AP) and significantly improving over MoCo-v2 on COCO detection (+1.0% APbb) and instance segmentation (+0.8% APmsk). Our synthetic hard negative generation approach significantly enhances visual representations learned through self-supervised contrastive learning. Code will be made publicly available.
URL: https://openreview.net/forum?id=yEyxDw11K8
---
Title: AB-UPT: Scaling Neural CFD Surrogates for High- Fidelity Automotive Aerodynamics Simulations via Anchored- Branched Universal Physics Transformers
Abstract: Recent advances in neural surrogate modeling offer the potential for transformative innovations in applications such as automotive aerodynamics. Yet, industrial-scale problems often involve volumetric meshes with cell counts reaching 100 million, presenting major scalability challenges. Complex geometries further complicate modeling through intricate surface-volume interactions, while quantities such as vorticity are highly nonlinear and must satisfy strict divergence-free constraints. To address these requirements, we introduce Anchored-Branched Universal Physics Transformers (AB-UPT) as a novel modeling scheme for building neural surrogates for computational fluid dynamics (CFD) simulations. AB-UPT is designed to: (i) decouple geometry encoding and prediction tasks via multi-branch operators; (ii) enable scalability to high-resolution outputs via neural simulation in a low-dimensional latent space, coupled with anchored neural field decoders to predict high-fidelity outputs; (iii) enforce physics consistency by a novel divergence-free formulation. We show that AB-UPT yields state-of-the-art predictive accuracy of surface and volume fields on automotive CFD simulations ranging from 33 thousand up to 150 million mesh cells. Furthermore, our anchored neural field architecture enables the enforcement of hard physical constraints on the physics predictions without degradation in performance, exemplified by modeling divergence-free vorticity fields. Notably, the proposed models can be trained on a single GPU in less than a day and predict industry-standard surface and volume fields within seconds. Additionally, we show that the flexible design of our method enables neural simulation from a computer-aided
design geometry alone, omitting the need for costly CFD meshing procedures.
URL: https://openreview.net/forum?id=nwQ8nitlTZ
---
Title: Know Yourself and Know Your Neighbour : A Syntactically Informed Self-Supervised Compositional Sentence Representation Learning Framework using a Recursive Hypernetwork
Abstract: In this work, we propose a syntactically informed self-supervised learning framework for generating sentence representations. In our framework, we train a recursive hypernetwork to compose sentence representation from any word-level representation by using a set of newly proposed self-supervised tasks. We verify that the newly proposed framework can generate sentence representations that encode more linguistic information than state-of-the-art sentence representations and verify the stability and adaptability of our model.
URL: https://openreview.net/forum?id=gfBiJv7r51
---
Title: Noise-Aware Adaptation of Vision Language Models for Single-photon Image Understanding
Abstract: Single-photon LiDAR enables high-resolution depth imaging under extreme photon-limited conditions, making it attractive for low-light and long-range 3D perception. Beyond depth reconstruction, semantic understanding from single-photon images remains challenging due to limited data and sensitivity to noise-induced appearance variation. In this work, we present a noise-aware adaptation framework that transfers large-scale vision-language models, such as CLIP, from natural RGB images to the novel modality of single-photon depth images for few-shot classification. We introduce a lightweight Noise Adapter that modulates CLIP visual features using summary statistics derived from raw single-photon histograms. This design helps decouple imaging noise from semantics, enabling more robust prediction under varying noise levels. Furthermore, we leverage the learned modulation pattern to guide feature-level augmentation, simulating feature changes caused by noise and improving generalization in the low-data regime. To the best of our knowledge, this is the first work to explicitly integrate noise-awareness into pre-trained model adaptation for single-photon images. Experiments on both synthetic and real single-photon datasets show that our method improves accuracy over baselines, with an average improvement of 3\% over the best baseline. These results highlight the importance of modeling physical noise in photon-limited imaging and demonstrate the potential of leveraging vision models pre-trained on conventional modalities to improve performance on single-photon depth data with limited supervision.
URL: https://openreview.net/forum?id=qSnrIy6Ohb
---
Title: SCNode: Spatial and Contextual Coordinates for Graph Representation Learning
Abstract: Effective node representation lies at the heart of Graph Neural Networks (GNNs), as it directly impacts their ability to perform downstream tasks such as node classification and link prediction. Most existing GNNs, particularly message passing graph neural networks, rely on neighborhood aggregation to iteratively compute node embeddings. While powerful, this paradigm suffers from well-known limitations of oversquashing, oversmoothing, and underreaching that degrade representation quality. More critically, MPGNNs often assume homophily, where connected nodes share similar features or labels, leading to poor generalization in heterophilic graphs where this assumption breaks down.
To address these challenges, we propose *SCNode*, a *Spatial-Contextual Node Embedding* framework designed to perform consistently well in both homophilic and heterophilic settings. SCNode integrates spatial and contextual information, yielding node embeddings that are not only more discriminative but also structurally aware. Our approach introduces new homophily matrices for understanding class interactions and tendencies. Extensive experiments on benchmark datasets show that SCNode achieves superior performance over conventional GNN models, demonstrating its robustness and adaptability in diverse graph structures.
URL: https://openreview.net/forum?id=wdcdKeFbfQ
---
Title: Temporal Test-Time Adaptation with State-Space Models
Abstract: Distribution shifts between training and test data are inevitable over the lifecycle of a deployed model, leading to performance decay. Adapting a model on test samples can help mitigate this drop in performance. However, most test-time adaptation methods have focused on synthetic corruption shifts, leaving a variety of distribution shifts underexplored. In this paper, we focus on distribution shifts that evolve gradually over time, which are common in the wild but challenging for existing methods, as we show. To address this, we propose STAD, a probabilistic state-space model that adapts a deployed model to temporal distribution shifts by learning the time-varying dynamics in the last set of hidden features. Without requiring labels, our model infers time-evolving class prototypes that act as a dynamic classification head. Through experiments on real-world temporal distribution shifts, we show that our method excels in handling small batch sizes and label shift.
URL: https://openreview.net/forum?id=HFETOmUtrV
---
Title: Mirror Descent Policy Optimisation for Robust Constrained Markov Decision Processes
Abstract: Safety is an essential requirement for reinforcement learning systems. The newly emerging framework of robust constrained Markov decision processes allows learning policies that satisfy long-term constraints while providing guarantees under epistemic uncertainty. This paper presents mirror descent policy optimisation for robust constrained Markov decision processes (RCMDPs), making use of policy gradient techniques to optimise both the policy (as a maximiser) and the transition kernel (as an adversarial minimiser) on the Lagrangian representing a constrained MDP. In the oracle-based RCMDP setting, we obtain an $\mathcal{O}\left(\frac{1}{T}\right)$ convergence rate for the squared distance as a Bregman divergence, and an $\mathcal{O}\left(e^{-T}\right)$ convergence rate for entropy-regularised objectives. In the sample-based RCMDP setting, we obtain an $\tilde{\mathcal{O}}\left(\frac{1}{T^{1/3}}\right)$ convergence rate. Experiments confirm the benefits of mirror descent policy optimisation in constrained and unconstrained optimisation, and significant improvements are observed in robustness tests when compared to baseline policy optimisation algorithms.
URL: https://openreview.net/forum?id=tmfdqtFUqO
---
Title: Bridging SFT and DPO for Diffusion Model Alignment with Self-Sampling Preference Optimization
Abstract: Existing post-training techniques are broadly categorized into supervised fine-tuning (SFT) and reinforcement learning (RL) methods; the former is stable during training but suffers from limited generalization, while the latter, despite its stronger generalization capability, relies on additional preference data or reward models and carries the risk of reward exploitation. In order to preserve the advantages of both SFT and RL -- namely, eliminating the need for paired data and reward models while retaining the training stability of SFT and the generalization ability of RL -- a new alignment method, Self-Sampling Preference Optimization (SSPO), is proposed in this paper. SSPO introduces a Random Checkpoint Replay (RCR) strategy that utilizes historical checkpoints to construct paired data, thereby effectively mitigating overfitting. Simultaneously, a Self-Sampling Regularization (SSR) strategy is employed to dynamically evaluate the quality of generated samples; when the generated samples are more likely to be winning samples, the approach automatically switches from DPO to SFT, ensuring that the training process accurately reflects the quality of the samples. Experimental results demonstrate that SSPO not only outperforms existing methods on text-to-image benchmarks, but its effectiveness has also been validated in text-to-video tasks. We validate SSPO across both text-to-image and text-to-video benchmarks. SSPO surpasses all previous approaches on the text-to-image benchmarks and demonstrates outstanding performance on the text-to-video benchmarks.
URL: https://openreview.net/forum?id=BawGeztQRz
---
Title: TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models
Abstract: Recent advances in multimodal Large Language Models (LLMs) have shown great success in understanding multimodal contents.
For video understanding tasks, training-based video LLMs are difficult to build due to the scarcity of high-quality, curated video-text paired data. In contrast, paired image-text data are much easier to obtain, and there is substantial similarity between images and videos. Consequently, extending image LLMs for video understanding tasks presents an appealing alternative. Developing effective strategies for compressing visual tokens from multiple frames is a promising way to leverage the powerful pre-trained image LLM. In this work, we explore the limitations of the existing compression strategies for building a training-free video LLM. The findings lead to our method TS-LLaVA, which constructs visual tokens through a Thumbnail-and-Sampling strategy. Given a video, we select few equidistant frames from all input frames to construct a Thumbnail image as a detailed visual cue, complemented by Sampled visual tokens from all input frames. Our method establishes the new state-of-the-art performance among training-free video LLMs on various benchmarks. Notably, our 34B model outperforms GPT-4V on the MVBench benchmark, and achieves performance comparable to the 72B training-based video LLM, Video-LLaMA2, on the challenging MLVU benchmark. We will release code upon acceptance.
URL: https://openreview.net/forum?id=ZRFNKyj43y
---
Title: SIPHER: Spike based Neuromorphic Computing for Secure Inference against Bit-Flip Attack
Abstract: Deep Artificial Neural Networks (ANNs) have been shown to be vulnerable to parameter attacks, such as the bit-flip attack, where intentional alterations of network weights can cause significant performance loss. Although extensive research has enhanced the efficacy of these attacks against standard ANN models, robust and efficient defense mechanisms remain underdeveloped. In this work, we propose the spike-based neuromorphic computing paradigm, referred to as SIPHER, as a potent defense strategy that exploits the inherent properties of Spiking Neural Networks (SNNs) to mitigate such attacks. SNNs have emerged as a biologically plausible and energy-efficient alternative to ANNs. However, their fault tolerance and robustness against parameter attacks have not yet been thoroughly investigated. We show that SNNs, on account of their temporal computing capability, effectively neutralize the state-of-the-art progressive bit search method for bit-flip attack, effectively rendering the attack equivalent to random bit-flips. Our results reveal that an 8-bit quantized ResNet-20 SNN requires 145$\times$ more malicious bit-flips compared to ANNs to achieve similar accuracy degradation, with 250$\times$ longer average attack time per bit-flip. The resilience of SNNs increases significantly with model size, with an 8-bit quantized VGG-16 SNN requiring 518$\times$ more bit-flips than ANNs to inflict comparable degradation, thus outperforming state-of-the-art defenses against bit-flip attack. We validate SIPHER on different models and datasets, thereby demonstrating the robustness of the spike-based inference method.
URL: https://openreview.net/forum?id=Btdz8Nbdnv
---
Title: BIM: Block-Wise Local Learning with Masked Image Modeling
Abstract: Like masked language modeling (MLM) in NLP, masked image modeling (MIM) extracts insights from image patches to enhance feature extraction in deep neural networks (DNNs). Unlike supervised learning, MIM pretraining requires substantial computational resources to handle large batch sizes (e.g., 4096), limiting its scalability. To address this, we propose Block-Wise Masked Image Modeling (BIM), which decomposes MIM tasks into sub-tasks with independent computations, enabling block-wise backpropagation instead of the traditional end-to-end approach. BIM achieves comparable performance to MIM while significantly reducing peak memory usage. For evaluation, we provide an anonymized GitHub repository ~\href{https://anonymous.4open.science/r/BIM_ICML2025/}{here}. Additionally, BIM facilitates concurrent training of multiple DNN backbones with varying depths, optimizing them for different hardware platforms while reducing computational costs compared to training each backbone separately.
URL: https://openreview.net/forum?id=I0i7rE25Cw
---
Title: AttentionBreaker: Adaptive Evolutionary Optimization for Unmasking Vulnerabilities in LLMs through Bit-Flip Attacks
Abstract: Large language models (LLMs) have significantly advanced natural language processing (NLP) yet are still susceptible to hardware-based threats, particularly bit-flip attacks (BFAs). Traditional BFA techniques, requiring iterative gradient recalculations after each bit-flip, become computationally prohibitive and lead to memory exhaustion as model size grows, making them impractical for state-of-the-art LLMs. To overcome these limitations, we propose AttentionBreaker, a novel framework for efficient parameter space exploration, incorporating GenBFA, an evolutionary optimization method that identifies the most vulnerable bits in LLMs. Our approach demonstrates unprecedented efficacy—flipping just three bits in the LLaMA3-8B-Instruct model, quantized to 8-bit weights (W8), completely collapses performance, reducing Massive Multitask Language Understanding (MMLU) accuracy from 67.3% to 0% and increasing Wikitext perplexity by a factor of $10^5$. Furthermore, AttentionBreaker circumvents existing defenses against BFAs on transformer-based architectures, exposing a critical security risk. Code is open sourced at: https://anonymous.4open.science/r/attention_breaker-16FF/.
URL: https://openreview.net/forum?id=2ekgTdBOZo
---
Title: D2 Actor Critic: Diffusion Actor Meets Distributional Critic
Abstract: We develop a new model-free reinforcement learning (RL) algorithm: D2AC. Motivated by recent advances in iterative function approximation, we make two adjustments to the typical actor-critic RL pipeline. First, we learn distributional critics with a novel fusion of distributional RL and clipped double Q-learning. Second, we use a diffusion model to parameterize the policy and derive an efficient method for aligning the diffusion process with policy improvement. These changes are highly effective, resulting in highly performant model-free policies on a benchmark of eighteen hard RL tasks, including Humanoid, Dog, and Shadow Hand domains, spanning both dense-reward and goal-conditioned RL scenarios. Beyond standard benchmarks, we also evaluate a biologically motivated predator-prey task to examine the behavioral robustness and generalization capacity of our approach.
URL: https://openreview.net/forum?id=8KbstCUXhH
---
Title: EL-Clustering: Clustering with Equitable Load Constraints
Abstract: The application of an ordinary clustering algorithm may result in a clustering output where the number of points per cluster (cluster size) varies widely. In settings where the centers correspond to facilities that provide a service this can be highly undesirable since the cluster size is essentially the service load for a facility. While prior work has considered imposing either a lower bound on the cluster sizes or an upper bound, imposing both bounds simultaneously has seen limited work especially for the $k$-median objective despite its strong practical motivation. In this paper we solve the \emph{equitable load} (EL) clustering problem where we minimize the $k$-median objective subject to the cluster sizes not exceeding an upper bound or falling below a lower bound. We solve this problem using a modular approach. Specifically, given a clustering solution that satisfies the lower bound constraints and another that satisfies the upper bound constraints, we introduce a combination algorithm which essentially combines both solutions to produce one that satisfies both constraints simultaneously at the expense of a bounded degradation in the $k$-median objective and a slight violation to the upper bound. Our combination algorithms runs in $O(k^3+n)$ time where $n$ is the number of points and is actually faster than standard $k$-median algorithms that satisfy either the lower or upper bound constraints. Interestingly, our results can be generalized to various other clustering objectives including the $k$-means objective.
URL: https://openreview.net/forum?id=EkjDfnJ1gU
---
Title: Context-aware Prompt Tuning: Enhancing Few-Shot Learning via Optimized Context Embeddings
Abstract: Large Language Models (LLMs) can perform few-shot learning using either In-Context Learning (ICL) or optimization-based methods.
While ICL typically excels in low-data regimes, optimization-based methods tend to perform better when more data is available.
This contrast raises an important question: Why do optimization-based methods struggle in low-data scenarios, and how can they be effectively combined with ICL to enhance few-shot learning? In this work, we identify overfitting as the primary limitation of optimization-based methods in few-shot learning. To address this, we propose Context-Aware Prompt Tuning (CPT), which extends ICL through a carefully designed optimization process specifically crafted to mitigate overfitting. CPT extracts richer insights from limited data while preserving the integrity of the original input samples. We validate our approach across diverse classification and question answering tasks and multiple LLM architectures. CPT consistently outperforms existing baselines across tasks and models, significantly reducing overfitting and improving generalization in few-shot scenarios.
URL: https://openreview.net/forum?id=V1YbbfyHPx
---
Title: Model Guidance via Robust Feature Attribution
Abstract: Controlling the patterns a model learns is essential to preventing reliance on irrelevant or misleading features. Such reliance on irrelevant features, often called shortcut features, has been observed across domains, including medical imaging and natural language processing, where it may lead to real-world harms. A common mitigation strategy leverages annotations (provided by humans or machines) indicating which features are relevant or irrelevant. These annotations are compared to model explanations, typically in the form of feature salience, and used to guide the loss function during training. Unfortunately, recent works have demonstrated that feature salience methods are unreliable and therefore offer a poor signal to optimize. In this work, we propose a simplified objective that simultaneously optimizes for explanation robustness and mitigation of shortcut learning. Unlike prior objectives with similar aims, we demonstrate theoretically why our approach ought to be more effective. Across a comprehensive series of experiments, we show that our approach consistently reduces test-time misclassifications by 20\% compared to state-of-the-art methods. We also extend prior experimental settings to include natural language processing tasks. Additionally, we conduct novel ablations that yield practical insights, including the relative importance of annotation quality over quantity. Code for our method and experiments is available at: https://anonymous.4open.science/r/ModelGuidanceViaRobustFeatureAttribution-7417.
URL: https://openreview.net/forum?id=AVAHxDSqUu
---
Title: An Unconditional Representation of the Conditional Score in Infinite Dimensional Linear Inverse Problems
Abstract: Score-based diffusion models (SDMs) have emerged as a powerful tool for sampling from the posterior distribution in Bayesian inverse problems. However, existing methods often require multiple evaluations of the forward mapping to generate a single sample, resulting in significant computational costs for large-scale inverse problems. To address this, we propose an unconditional representation of the conditional score-function (UCoS) tailored to linear inverse problems, which avoids forward model evaluations during sampling by shifting computational effort to an offline training phase. In this phase, a task-dependent score function is learned based on the linear forward operator. Crucially, we show that the conditional score can be derived exactly from a trained (unconditional) score using affine transformations, eliminating the need for conditional score approximations. Our approach is formulated in infinite-dimensional function spaces, making it inherently discretization-invariant. We support this formulation with a rigorous convergence analysis that justifies UCoS beyond any specific discretization. Finally we validate UCoS through high-dimensional computed tomography (CT) and image deblurring experiments, demonstrating both scalability and accuracy.
URL: https://openreview.net/forum?id=rO8erhXHPo
---
Title: Tracking the Median of Gradients with a Stochastic Proximal Point Method
Abstract: There are several applications of stochastic optimization where one can benefit from a robust estimate of the gradient. For example, domains such as distributed learning with corrupted nodes, the presence of large outliers in the training data, learning under privacy constraints, or even heavy-tailed noise due to the dynamics of the algorithm itself. Here we study SGD with robust gradient estimators based on estimating the median.
We first derive iterative methods based on the stochastic proximal point method for computing the median gradient and generalizations thereof. Then we propose an algorithm estimating the median gradient across *iterations*, and find that several well known methods are particular cases of this framework.
For instance, we observe that different forms of clipping allow to compute online estimators of the *median* of gradients, in contrast to (heavy-ball) momentum, which corresponds to an online estimator of the *mean*.
Finally, we provide a theoretical framework for any algorithm computing the median gradient across *samples*, and show that the resulting method can converge even under heavy-tailed, state-dependent noise.
URL: https://openreview.net/forum?id=WMxLEgYGxu
---
Title: PROPS: Progressively Private Self-alignment of Large Language Models
Abstract: Alignment is a key step in developing Large Language Models (LLMs) using human feedback to ensure adherence to human values and societal norms. Dependence on human feedback raises privacy concerns about how much a labeler’s preferences may reveal about their personal values, beliefs, and personality traits. Existing approaches, such as Differentially Private SGD (DP-SGD), provide rigorous privacy guarantees by privatizing gradients during fine-tuning and alignment but can provide more privacy than necessary as human preferences are tied only to labels of (prompt, response) pairs and can degrade model utility. This work focuses on LLM alignment with preference-level privacy, which preserves the privacy of preference labels provided by humans. We propose PROPS (PROgressively Private Self-alignment), a multi-stage privacy preserving alignment framework where privately aligned models in previous stages can serve as labelers for supplementing training data in the subsequent stages of alignment. We present theoretical guarantees for PROPS as well as comprehensive validation using multiple models (Pythia and GPT) and datasets (AlpacaEval, Anthropic HH-RLHF, truthy-dpo-v0.1) to demonstrate the utility of PROPS over existing methods while still providing high privacy. For the same privacy budget, alignment via PROPS can achieve up to 3x higher win-rates compared to DP-SGD, and 2.5x higher win-rates compared to Randomized Response (RR) based alignment.
URL: https://openreview.net/forum?id=phbRwhaeBo
---
Title: Simplifying Knowledge Transfer in Pretrained Models
Abstract: Pretrained models are ubiquitous in the current deep learning landscape, offering strong results on a broad range of tasks. Recent works have shown that models differing in various design choices exhibit categorically diverse generalization behavior, resulting in one model grasping distinct data-specific insights unavailable to the other. In this paper, we propose to leverage large publicly available model repositories as an auxiliary source of model improvements. We introduce a data partitioning strategy where pretrained models autonomously adopt either the role of a student, seeking knowledge, or that of a teacher, imparting knowledge, fostering a collaborative learning environment. Experiments across various tasks demonstrate the effectiveness of our proposed approach. In image classification, we improved the performance of ViT-B by approximately 1.4\% through bidirectional knowledge transfer with ViT-T. For semantic segmentation, our method boosted all evaluation metrics by enabling knowledge transfer both within and across backbone architectures. In video saliency prediction, our approach achieved a new state-of-the-art. We further extend our approach to knowledge transfer between multiple models, leading to considerable performance improvements for all model participants.
URL: https://openreview.net/forum?id=eQ9AVtDaP3
---
Title: Geometric Optimal Transport for Unsupervised Domain Adaptation
Abstract: Optimal Transport (OT) is a widely used and powerful approach in domain adaptation.
While effective, most existing methods rely on the pairwise squared Euclidean distances for the transportation cost, implicitly assuming a Euclidean space.
In this paper, we challenge this assumption by introducing Geometric Optimal Transport (GOT), a new transport cost designed for domain adaptation under the manifold assumption.
By utilizing concepts and tools from the field of manifold learning, specifically diffusion geometry, we derive an operator that accounts for the intra-domain geometries, extending beyond the conventional inter-domain distances.
This operator, which quantifies the probability of transporting between source and target samples, forms the basis for our cost.
We demonstrate how the proposed cost, defined by an anisotropic diffusion process, naturally aligns with the desired properties for domain adaptation.
To further enhance performance, we integrate source labels into the operator, thereby guiding the anisotropic diffusion according to the classes.
We showcase the effectiveness of GOT through comprehensive experiments, demonstrating its superior performance compared to recent methods across various benchmarks and datasets.
URL: https://openreview.net/forum?id=8Nef4vZUzU
---
Title: On Low Frequencies Fourier Features in Reinforcement Learning
Abstract: Recent reinforcement learning (RL) methods have adopted ideas from image processing tasks by employing Fourier Features (FFs) encoding. This approach enables a typical multilayer perceptron (MLP) to learn different frequency features. However, a disparity exists between the scale of frequencies used for image and RL problems. Previous works employed significant lower frequencies to successfully train RL agents and defer to the Neural Tangent Kernels (NTK) theory for justification. However, we observed that NTK cannot provide satisfactory explanations. We present a novel perspective empirically to show why lower frequencies are essential for the successful training of RL agents. Our empirical investigation is based on the cross-correlation among state dimensions and their overall cross energy spectral density (CSD). Based on our empirical observation, we propose a simple enhancement to the current FFs formulation and achieve performance improvements over current FFs formulation and baseline methods.
URL: https://openreview.net/forum?id=LDCsN3SGXA
---
Title: Bayesian Optimization over Discrete Structured Inputs by Continuous Objective Relaxation
Abstract: To optimize efficiently over discrete data from few available target observations is a challenge in Bayesian optimization. We propose a continuous relaxation of the objective function and show that inference and optimization is computationally tractable. The advantages are the continuous treatment of the problem and directly incorporating available prior knowledge over the inputs. Motivated by optimizing expensive biochemical properties from discrete sequences, we consider optimization with few observations and strict budgets. We leverage available and learned distributions from domain models for a weighting of the Hellinger distance, which we show to be a covariance function. Our results include a domain-model likelihood weighted kernel and acquisition function optimization with continuous and discrete algorithms. Lastly, we compare against state-of-the-art Bayesian optimization algorithms on sequence optimization tasks: 25 small-molecule tasks and two protein objectives.
URL: https://openreview.net/forum?id=m0FL1EHxlK
---
Title: ExDBN: Learning Dynamic Bayesian Networks using Extended Mixed-Integer Programming Formulations
Abstract: Causal learning from data has received much attention recently. Bayesian networks can be used to capture causal relationships. There, one recovers a weighted directed acyclic graph in which random variables are represented by vertices, and the weights associated with each edge represent the strengths of the causal relationships between them.
This concept is extended to capture dynamic effects by introducing a dependency on past data, which may be captured by the structural equation model. This formalism is utilized in the present contribution to propose a score-based learning algorithm. A mixed-integer quadratic program is formulated and an algorithmic solution proposed, in which the pre-generation of exponentially many acyclicity constraints is avoided by utilizing the so-called branch-and-cut (``lazy constraint'') method.
Comparing the novel approach to the state-of-the-art, we show that the proposed approach turns out to produce more accurate results when applied to small and medium-sized synthetic instances containing up to 25 time series. Lastly, two interesting applications in bioscience and finance, to which the method is directly applied, further stress the importance of developing highly accurate, globally convergent solvers that can handle instances of modest size.
URL: https://openreview.net/forum?id=I64MJzl9Fy
---
Title: Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?
Abstract: As large language models (LLMs) continue to evolve, ensuring their alignment with human goals and values remains a pressing challenge. A key concern is instrumental convergence, where an AI system, in optimizing for a given objective, develops unintended intermediate goals that override the ultimate objective and deviate from human-intended goals. This issue is particularly relevant in reinforcement learning (RL)-trained models, which can generate creative but unintended strategies to maximize rewards. In this paper, we explore instrumental convergence in LLMs by comparing models trained with direct RL optimization (e.g., the o1 model) to those trained with reinforcement learning from human feedback (RLHF). We hypothesize that RL-driven models exhibit a stronger tendency for instrumental convergence due to their optimization of goal-directed behavior in ways that may misalign with human intentions. To assess this, we introduce InstrumentalEval, a benchmark for evaluating instrumental convergence in RL-trained LLMs. Initial experiments reveal cases where a model tasked with making money unexpectedly pursues instrumental objectives, such as self-replication, implying signs of instrumental convergence. Our findings contribute to a deeper understanding of alignment challenges in AI systems and the risks posed by unintended model behaviors.
URL: https://openreview.net/forum?id=Lw1m3lXYzW
---
Title: Estimating the Event-Related Potential from Few EEG Trials
Abstract: Event-related potentials (ERP) are measurements of brain activity with wide applications in basic and clinical neuroscience, that are typically estimated using the average of many trials of electroencephalography signals (EEG) to sufficiently reduce noise and signal variability.
We introduce EEG2ERP, a novel uncertainty-aware autoencoder approach that maps an arbitrary number of EEG trials to their associated ERP. To account for the ERP uncertainty we use bootstrapped training targets and introduce a separate variance decoder to model the uncertainty of the estimated ERP.
We evaluate our approach in the challenging zero-shot scenario of generalizing to new subjects considering three different publicly available data sources; i) the comprehensive ERP CORE dataset that includes over 50,000 EEG trials across six ERP paradigms from 40 subjects, ii) the large P300 Speller BCI dataset, and iii) a neuroimaging dataset on face perception consisting of both EEG and magnetoencephalography (MEG) data. We consistently find that our method in the few trial regime provides substantially better ERP estimates than commonly used conventional and robust averaging procedures.
EEG2ERP is the first deep learning approach to map EEG signals to their associated ERP, moving toward reducing the number of trials necessary for ERP research.
URL: https://openreview.net/forum?id=c6LgqDhpH0
---
Title: Kernel Space Conditional Distribution Alignment for Improving Group Fairness in Deepfake Detection
Abstract: We introduce FairAlign, a new method to reduce bias and improve group fairness in deepfake detection by aligning conditional distributions of embeddings in a high-dimensional kernel space. Our approach reduces information related to sensitive attributes in the embedding space that could potentially bias the detection process, thus promoting fairness. FairAlign is a versatile plug-and-play loss term compatible with various deepfake detection networks and is capable of enhancing group fairness without compromising detection performance. In addition to applying FairAlign for reducing gender bias, we implement a systematic pipeline for the annotation of skin tones and promotion of fairness in deepfake detection related to this sensitive attribute. Finally, we perform the first comprehensive study toward quantifying and understanding the trade-off between fairness and accuracy in the context of deepfake detection. We use three public deepfake datasets FaceForensics++, CelebDF, and WildDeepfake to evaluate our method. Through various experiments, we observe that FairAlign outperforms other bias-mitigating methods across various deepfake detection backbones for both gender and skin tone, setting a new state-of-the-art. Moreover, our fairness-accuracy trade-off analysis demonstrates that our approach demonstrates the best overall performance when considering effectiveness in both deepfake detection and reducing bias. We release the code at: https://anonymous.4open.science/r/FairAlign-170F.
URL: https://openreview.net/forum?id=68Lv6v4N9J
---
Title: FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformers
Abstract: Diffusion Models (DM) have revolutionized the text-to-image visual generation process. However, the large computational cost and model footprint of DMs hinders practical deployment, especially on edge devices. Post-training quantization (PTQ) is a lightweight method to alleviate these burdens without the need for training or fine-tuning. While recent DM PTQ methods achieve W4A8 on integer-based PTQ, two key limitations remain: First, while most existing DM PTQ methods evaluate on classical DMs like Stable Diffusion XL, 1.5 or earlier, which use convolutional U-Nets, newer Diffusion Transformer (DiT) models like the PixArt series, Hunyuan and others adopt fundamentally different transformer backbones to achieve superior image synthesis. Second, integer (INT) quantization is prevailing in DM PTQ but does not align well with the network weight and activation distribution, while Floating-Point Quantization (FPQ) is still under-investigated, yet it holds the potential to better align the weight and activation distributions in low-bit settings for DiT. In this paper, we introduce FP4DiT, a PTQ method that leverages FPQ to achieve W4A6 quantization. Specifically, we extend and generalize the Adaptive Rounding PTQ technique to adequately calibrate weight quantization for FPQ and demonstrate that DiT activations depend on input patch data, necessitating robust online activation quantization techniques. Experimental results demonstrate that FP4DiT outperforms integer-based PTQ at W4A6 and W4A8 precision and generates convincing visual content on PixArt-$\alpha$, PixArt-$\Sigma$ and Hunyuan in terms of several T2I metrics such as HPSv2 and CLIP.
URL: https://openreview.net/forum?id=CcnH4mSQbP
---
Title: A Note On The Stability Of The Focal Loss
Abstract: The Focal loss is a widely deployed loss function that is used to train various types of deep learning models. This loss function is a modification of the cross-entropy loss designed to mitigate the effect of class imbalance in dense object detection tasks by downweighing
easy, well-classified examples. In doing so, more focus is placed on hard, wrongly-classified examples by preventing the gradients from being dominated by examples from which the model can easily predict the correct class. This downweighing is achieved by scaling the
cross-entropy loss with a term that depends on a focusing parameter $\gamma$. In this paper, we highlight an unaddressed instability of the Focal loss that arises when this focusing parameter is set to a value between 0 and 1. We present the theoretical foundation behind
this instability, show that it is numerically identifiable, and demonstrate it in a binary classification and segmentation task on the MNIST dataset. Additionally, we propose a straightforward modification to the original Focal loss to ensure stability whenever these unstable focusing parameter values are used.
URL: https://openreview.net/forum?id=eCYActnGbu
---
Title: DNOD: Deformable Neural Operators for Object Detection in SAR Images
Abstract: We introduce a deep neural operator framework aimed at object detection in remotely sensed Synthetic Aperture Radar (SAR) images. Recent research highlights the impressive performance of the End-to-End Object Detection Transformer (DETR). Nonetheless, in domains like SAR imaging, managing challenges such as speckle noise and the detection of small objects continues to be problematic. To address SAR object detection issues, we present the Deformable Neural Operator-Based Object Detection (DNOD) framework, tailored for SAR tasks. We develop two neural operators: Multi-Scale Fourier Mixing (MSFM) for the encoder and Multi-scale, multi-input Adaptive Deformable Fourier Neural Operator (MADFNO) for the decoder. Detailed evaluations and ablation studies show that DNOD exceeds existing methods, delivering significantly better results with an improvement of +2.23 mAP on the SARDet-100k dataset, the largest SAR object detection compilation.
URL: https://openreview.net/forum?id=tjBqPJdQ72
---
Title: MMD Two-sample Testing in the Presence of Arbitrarily Missing Data
Abstract: In many real-world applications, it is common that a proportion of the data may be missing or only partially observed. We develop a novel two-sample testing method based on the Maximum Mean Discrepancy (MMD) which accounts for missing data in both samples, without making assumptions about the missingness mechanism. Our approach is based on deriving the mathematically precise bounds of the MMD test statistic after accounting for all possible missing values. To the best of our knowledge, it is the only two-sample testing method that is guaranteed to control the Type I error for both univariate and multivariate data where data may be arbitrarily missing. Simulation results show that the method has good statistical power, typically for cases where 5% to 10% of the data are missing. We highlight the value of this approach when the data are missing not at random, a context in which either ignoring the missing values or using common imputation methods may not control the Type I error.
URL: https://openreview.net/forum?id=GfcDel1ICb
---
Title: Soup to go: mitigating forgetting during continual learning with model averaging
Abstract: In continual learning, where task data arrives in a sequence, fine-tuning on later tasks will often lead to performance degradation on earlier tasks. This is especially pronounced when these tasks come from diverse domains. In this setting, how can we mitigate catastrophic forgetting of earlier tasks and retain what the model has learned with minimal computational expenses? We propose Sequential Fine-tuning with Averaging (SFA), a method that merges currently training models with earlier checkpoints during the course of training. Our method outperforms SOTA merging, and penalty methods while achieving comparable results to maintaining a data buffer of past tasks without actually storing past data. In turn, our method offers insight into the benefits of merging partially trained models during training across both image and language domains.
URL: https://openreview.net/forum?id=khETLNdevT
---
Title: SURE: Scalable Uncertainty Estimation for Multimodal Pretrained Pipelines with Missing Inputs
Abstract: Pretrained multimodal models offer strong representational priors and sample efficiency, but remain fragile when deployed in real-world settings. Two key challenges underlie this brittleness: (1) inputs are frequently incomplete due to missing or corrupted modalities, and (2) pretrained models may yield unreliable predictions due to distribution mismatch or insufficient adaptation. A common workaround for the first challenge is to reconstruct missing modalities; however, this alone not only fails to resolve the second challenge, but may exacerbate it--introducing additional uncertainty from reconstruction that compounds the inherent unreliability of the pretrained model. We propose \textbf{SURE} (Scalable Uncertainty and Reconstruction Estimation), a lightweight, plug-and-play module that enhances pretrained multimodal pipelines with deterministic latent-space reconstruction and principled uncertainty estimation. SURE decomposes prediction uncertainty into two sources: \textit{input-induced uncertainty}, traced from reconstruction via error propagation, and \textit{model mismatch uncertainty}, reflecting the limits of the frozen model. To support stable uncertainty learning, SURE employs a distribution-free Pearson correlation-based loss that aligns uncertainty scores with reconstruction and task errors. Evaluated on both a tractable linear-Gaussian toy problem and several real-world tasks, SURE improves prediction accuracy and uncertainty calibration, enabling robust, trust-aware inference under missing or unreliable input conditions.
URL: https://openreview.net/forum?id=G4EzUnMhnQ
---
Title: One-Sided Matrix Completion from Ultra-Sparse Samples
Abstract: Matrix completion is a classical problem that has received recurring interest from a wide range of fields. In this paper, we revisit matrix completion in an ultra-sparse sampling setting, where each entry of an unknown, $n$ by $d$ matrix $M$, is observed with probability $p = \frac C d$, for any constant $C \ge 2$ (assuming $n \ge d$). This setting is motivated by large-scale panel datasets with high sparsity in practice. While the total number of observed samples, or roughly $C n$, is insufficient to recover $M$, we show that it is possible to recover one side of $M$, i.e., the second-moment of the row vectors, given by $T = \frac 1 n M^{\top} M$. The empirical second moment computed from observational data involves non-random missingness and high sparsity. We design an algorithm that estimates $T$ by normalizing every nonzero entry of the empirical second moment with its observed frequency, followed by gradient descent to impute the missing entries. The normalized entry divides a weighted sum of $n$ binomial random variables by the total number of ones, which is challenging to analyze due to nonlinearity and sparsity. We provide estimation and recovery guarantees for this estimator in the ultra-sparse regime, showing that it is unbiased for any $p$, and incurs low variance. Assuming the row vectors of $M$ are sampled from a rank-$r$ factor model, we prove that when $n \ge O(\frac{d r^5 \log d}{C^2\epsilon^2})$, our algorithm can recover $T$ with Frobenius norm error less than $\epsilon^2$, assuming the rank-$r$ factor model satisfies a standard incoherence condition. We also extend the use of one-sided matrix completion as a sub-procedure towards imputing the missing entries of $M$.
Experiments on both synthetic and real-world data are provided to evaluate this approach. When tested on three MovieLens datasets, our approach reduces bias by $88\%$ relative to its alternatives. We also validate the linear sampling complexity of $n$ relative to $d$ on synthetic data. On an Amazon reviews dataset with sparsity $10^{-7}$, our approach reduces the recovery error of $T$ by $59\%$ and $M$ by $38\%$ compared to existing matrix completion methods.
URL: https://openreview.net/forum?id=vYGi4Dj777
---
Title: Benchmarking Text-to-Image Safety: Using Adaptation Methods to Mitigate Oversexualization
Abstract: Generative text-to-image (T2I) models are capable of producing high quality images from user prompts. However, these models are known to generate sexually explicit content even for benign prompts, posing safety risks and misaligning with user intent. While emerging research proposes mitigation techniques to reduce sexually explicit content, there has yet to be a systematic benchmark to evaluate their effectiveness. Furthermore, little attention has been paid to oversexualization, cases where the generated images are more sexualized than the user prompt intends, which presents a distinct safety risk. Oversexualization may have more adverse outcomes than intentional adversarial prompting as it leaves users unintentionally exposed to harmful content. In this paper, we introduce the first comprehensive benchmark of adaptation methods, including both inference-time and fine-tuning methods, to mitigate oversexualized content in T2I models. We also introduce a novel benchmark dataset, Benign2NSFW, designed to provoke oversexualization in T2I systems, to allow the community to measure the effectiveness of such techniques. Finally,we assess the impact of reducing oversexualization on other factors, such as aesthetic quality and image-prompt alignment. Our work offers a comprehensive overview of various strategies for harm reduction in T2I systems, which we hope will help practitioners balance safety with other quality aspects.
URL: https://openreview.net/forum?id=FxX6g2aQZO
---
Title: Before Forgetting, There's Learning: Representation Learning Challenges in Online Unsupervised Continual Learning
Abstract: This paper addresses the Online Continual Unsupervised Learning (O-UCL) problem, where a learner must adapt to a stream of data arriving sequentially from a shifting distribution without storing past data or relying on labels. This challenge mirrors many real-world machine learning applications, where efficient training and updating of large or on device models is critical. We first explore the unique challenges of O-UCL and identify a secondary failure mode in addition to catastrophic forgetting. We demonstrate that the presence of transient, small-scale biases in an online data stream can significantly impair learning. Unlike traditional notions of distribution shift that manifest over long timescales, we highlight how biases occurring at the level of individual batches or short segments—while imperceptible in aggregate—can severely hinder a model’s ability to learn, a phenomenon we call ``catastrophic non-learning''. We further showcase how an auxiliary memory can be used to solve both catastrophic forgetting and catastrophic non-learning, but that the criteria for the ideal memory for each are in conflict. In response to these findings, we introduce a dual-memory framework which incorporates specifically designed modules to mitigate both catastrophic non-learning and forgetting. We validate our findings on challenging, realistic data streams derived from ImageNet and Places365, comparing against multiple baselines to highlight the distinct nature of this problem and the need for new approaches in O-UCL.
URL: https://openreview.net/forum?id=hZwInyuYDw
---