Weekly TMLR digest for May 03, 2026
TMLR
New certifications
==================
Featured Certification, J2C Certification: MixtureVitae: Open Web-Scale Pretraining Dataset With High Quality Instruction and Reasoning Data Built from Permissive-First Text Sources
Huu Nguyen, Victor May, Harsh Raj, Marianna Nezhurina, Yishan Wang, Yanqi Luo, Vu Minh Chien, Taishi Nakamura, Ken Tsui, Van Khue Nguyen, David Salinas, Aleksandra Krasnodębska, Christoph Schuhmann, Mats Leon Richter, Xuan-Son Vu, Jenia Jitsev
https://openreview.net/forum?id=SyCcUNUUMf
---
Reproducibility Certification: Pull-to-Outlier & Contrastive Objective-level (POCO) Unlearning: A Framework for Sample and Objective Forgetting
Agil Aghasanli, Plamen P Angelov
https://openreview.net/forum?id=KQxEwiA0VE
---
J2C Certification: Hierarchical Filtering and Refinement Classification for Few-Shot Class-Incremental Learning
Li-Jun Zhao, Zhen-Duo Chen, Xin Luo, Xin-Shun Xu
https://openreview.net/forum?id=7MXra1JSh8
---
J2C Certification: SAMAT: A Stereotype-Aware Multimodal Transformer for Interpretable Misogynistic Meme Detection
gopendra Vikram singh, Arpan Phukan, Kushal Kanwar, Asif Ekbal
https://openreview.net/forum?id=1DJBFS7rLW
---
J2C Certification: Riemannian Generative Decoder
Andreas Bjerregaard, Søren Hauberg, Anders Krogh
https://openreview.net/forum?id=vuPMXg1FDT
---
Accepted papers
===============
Title: Augmented Mixup Procedure for Privacy-Preserving Collaborative Training
Authors: Mihail-Iulian Pleșa, Fabrice Clérot, Simona Elena David, Robert Poenaru
Abstract: Mixup involves training neural networks on convex combinations of input samples and labels and has been adapted for privacy-preserving collaborative training, most notably in InstaHide. However, mixing-based obfuscation schemes create structured linear systems that can be exploited to reconstruct the underlying private data. We propose a singularized mixup procedure that injects controlled perturbations prior to forming convex combinations, rendering the resulting inverse problem ill-conditioned while preserving discriminative structure. We provide an average-case theoretical analysis that characterizes the security--utility trade-off via minimax reconstruction bounds and directional signal-to-noise ratio control. Empirically, we evaluate classification accuracy on MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet, and compare against InstaHide, observing competitive or improved accuracy under strong privacy settings. We assess robustness against both linear and nonlinear reconstruction attacks, including at-scale linear inversion experiments on CIFAR-5M. In a collaborative training setting with multiple parties and heterogeneous data partitions, we further compare against standard federated learning (FedProx), showing that singularized mixup enables accurate centralized training without iterative gradient exchange and yields improved robustness and performance in heterogeneous regimes. Overall, our results demonstrate that singularized mixup substantially degrades reconstruction quality while maintaining strong predictive performance, providing a practical and scalable approach to privacy-preserving collaborative learning.
URL: https://openreview.net/forum?id=1SrZyNgmpY
---
Title: Make Your LVLM KV Cache More Lightweight
Authors: Xihao Chen, Yangyang Guo, Roger Zimmermann
Abstract: Key-Value (KV) cache has become a de facto component of modern Large Vision-Language Models (LVLMs) for inference. While it enhances decoding efficiency in Large Language Models (LLMs), its direct adoption in LVLMs introduces substantial GPU memory overhead due to the large number of vision tokens processed during the prefill stage. To tackle this problem, we propose LightKV, a novel approach that reduces KV cache size by exploiting the redundancy among vision-token embeddings. Guided by text prompts, LightKV employs cross-modality message passing to aggregate informative messages across vision tokens and progressively compress them during prefill. This prompt-aware guidance distinguishes our method from prior vision-only compression strategies. We evaluate LightKV on eight open-source LVLMs across eight public benchmark datasets, e.g., MME and SeedBench. Experimental results demonstrate that with only 55% of the original vision tokens, LightKV (a) halves the vision-token KV cache size, (b) reduces computation by up to 40%, and (c) preserves general-purpose performance while significantly outperforming existing baselines.
URL: https://openreview.net/forum?id=n77IeySrQl
---
Title: Semantic-Drive: Trustworthy and Efficient Long-Tail Data Curation via Open-Vocabulary Grounding and Neuro-Symbolic VLM Consensus
Authors: Antonio Guillen-Perez
Abstract: The development of Autonomous Vehicles (AVs) is currently hampered by a scarcity of long-tail training data. While fleets collect petabytes of video logs, identifying rare safety-critical events, specifically scenarios like erratic jaywalking or complex construction diversions, remains a manual process that is often cost-prohibitive. Existing automated solutions rely either on coarse metadata search, which lacks semantic precision, or on cloud-based Vision-Language Models (VLMs), which introduce privacy concerns and computational overhead. In this work, we introduce Semantic-Drive, a local-first, neuro-symbolic framework designed for verifiable semantic data mining. Our approach decouples perception into two distinct stages: (1) Symbolic Grounding via a real-time open-vocabulary detector (YOLOE) to anchor attention, and (2) Cognitive Analysis, where a Reasoning VLM performs forensic scene analysis. To reduce hallucinations and reliability issues common in generative models, we implement a "System 2" inference-time alignment strategy that utilizes a multi-model "Judge-Scout" consensus mechanism. When benchmarked on the nuScenes dataset against the Waymo Open Dataset (WOD-E2E) taxonomy, it was observed that Semantic-Drive achieves a recall of 0.966 on safety-critical scenarios (vs. 0.331 for OWL-v2 and 0.271 for Grounding DINO). Notably, the system reduces risk assessment error by 40% compared to single-model baselines. The entire pipeline runs on consumer hardware (NVIDIA RTX 3090), offering an accessible and privacy-preserving alternative to cloud-native architectures.
URL: https://openreview.net/forum?id=qN2oN36L3k
---
Title: MixtureVitae: Open Web-Scale Pretraining Dataset With High Quality Instruction and Reasoning Data Built from Permissive-First Text Sources
Authors: Huu Nguyen, Victor May, Harsh Raj, Marianna Nezhurina, Yishan Wang, Yanqi Luo, Vu Minh Chien, Taishi Nakamura, Ken Tsui, Van Khue Nguyen, David Salinas, Aleksandra Krasnodębska, Christoph Schuhmann, Mats Leon Richter, Xuan-Son Vu, Jenia Jitsev
Abstract: We present MixtureVitae, an open‑access pretraining corpus built to minimize legal risk while providing strong downstream performance. MixtureVitae follows a permissive‑first, risk‑mitigated sourcing strategy that combines public‑domain and permissively licensed text (e.g., CC‑BY/Apache) with carefully justified low‑risk additions (e.g., government works and EU TDM‑eligible sources). MixtureVitae adopts a simple, single-stage pretraining recipe that integrates a large proportion of permissive synthetic instruction and reasoning data—signals typically introduced during post-training and generally scarce in permissive web corpora. We categorize all sources into a three-tier scheme that reflects varying risk levels and provide shard-level provenance metadata to enable risk-aware usage. In controlled experiments using the open‑sci‑ref training protocol (fixed architectures and hyperparameters; 50B and 300B token budgets across 130M–1.7B parameters), models trained on MixtureVitae consistently outperform other permissive datasets across a suite of standard benchmarks, and at the 1.7B-parameters/300B-tokens setting, they match FineWeb‑Edu and approach DCLM--demonstrating that the large fraction of reasoning and instruction data does not come at the cost of general-purpose language understanding. Performance is particularly strong on MMLU and on math and code benchmarks: a 1.7B model pretrained on 300B MixtureVitae tokens outperforms all strong non-permissive reference datasets and matches or exceeds smolLM2-Instruct, a strong 1.7B instruction‑tuned baseline on GSM8K, HumanEval, and MBPP, despite using over 36$\times$ fewer tokens (300B vs. $\approx$11T). Supported by a thorough decontamination analysis, these results show that permissive‑first data with high instruction and reasoning density, tiered by licensing and provenance-related risk, can provide a practical and risk-mitigated foundation for training capable LLMs, reducing reliance on broad web scrapes without sacrificing competitiveness. Dataset, source code for experiments reproduction and pre-trained models are available at https://github.com/ontocord/mixturevitae .
URL: https://openreview.net/forum?id=SyCcUNUUMf
---
Title: FL-Sailer: Efficient and Privacy-Preserving Federated Learning for Scalable Single-Cell Epigenetic Data Analysis via Adaptive Sampling
Authors: Guangyi Zhang, Yi Dai, Yiyun He, Junhao Liu
Abstract: Single-cell ATAC-seq (scATAC-seq) enables high-resolution mapping of chromatin accessibility, yet privacy regulations and data size constraints hinder multi-institutional sharing. Federated learning (FL) offers a privacy-preserving alternative, but faces three fundamental barriers in scATAC-seq analysis: ultra-high dimensionality, extreme sparsity, and severe cross-institutional heterogeneity. We propose FL-Sailer, the first FL framework designed for scATAC-seq data. FL-Sailer integrates two key innovations: (i) adaptive leverage score sampling, which selects biologically interpretable features while reducing dimensionality by 80%, and (ii) an invariant VAE architecture, which disentangles biological signals from technical confounders via mutual information minimization. We provide a convergence guarantee, showing that FL-Sailer converges to an approximate solution of the original high-dimensional problem with bounded error. Extensive experiments on synthetic and real epigenomic datasets demonstrate that FL-Sailer not only enables previously infeasible multi-institutional collaborations but also surpasses centralized methods by leveraging adaptive sampling as an implicit regularizer to suppress technical noise. Our work establishes that federated learning, when tailored to domain-specific challenges, can become a superior paradigm for collaborative epigenomic research.
URL: https://openreview.net/forum?id=2vNebz5r4b
---
Title: CauFR-TS: Causal Time-Series Identifiability via Factorized Representations
Authors: Ayanabha Ghosh, Debasis Das, Asif Ekbal
Abstract: Causal discovery from multivariate time series is a fundamental problem for interpretable modelling, causality-aware downstream analysis, and intervention-driven simulation. Recent neural approaches commonly rely on shared latent embeddings to capture temporal dynamics and utilize them for causal structure estimation and downstream prediction. We formally establish that such shared encoders entangle distinct causal mechanisms into a unified latent manifold, which exhibits fundamental theoretical limitations of structural non-identifiability and conditional independence assumptions required for Granger causality. To address these issues, we propose CauFR-TS, a recurrent variational framework that enforces mechanism modularity through dimension-wise encoders and ensures mediation of all cross-variable dependencies through structured latent aggregation. Furthermore, we address the instability of heuristic thresholding in continuous relaxation methods by proposing an adaptive, data-driven unsupervised link selection strategy based on decoder weight distribution. Empirical evaluation on synthetic and in silico biological benchmarks demonstrates that CauFR-TS outperforms recent baselines in graph recovery metrics while preserving competitive probabilistic forecasting performance.
URL: https://openreview.net/forum?id=Al4OnLoQsp
---
Title: Overcoming Output Dimension Collapse: When Sparsity Enables Zero-shot Brain-to-image Reconstruction at Small Data Scales
Authors: Kenya Otsuka, Yoshihiro Nagano, Yukiyasu Kamitani
Abstract: Advances in brain-to-image reconstruction are enabling us to externalize the subjective visual experiences encoded in the brain as images. A key challenge in this task is data scarcity: a translator that maps brain activity to latent image features is trained on a limited number of brain-image pairs, making the translator a bottleneck for zero-shot reconstruction beyond the training stimuli. In this paper, we mathematically analyze the behavior of two translators commonly used in recent reconstruction pipelines: naive multivariate linear regression and sparse multivariate linear regression. We define the data scale as the ratio of the number of training samples to the latent feature dimensionality and characterize the behavior of each model across data scales. Building on a standard structural property of naive multivariate regression, we first show that the resulting ''output dimension collapse'' can become a practical generalization bottleneck in brain-to-image reconstruction. We introduce the best prediction diagnostic, which is computable without brain activity, to quantify the practical impact of this collapse. We then analyze sparse linear regression models in a student--teacher framework and derive expressions for the prediction error in terms of data scale and other sparsity-related parameters. Our analysis clarifies when variable selection can reduce prediction error at small data scales by exploiting the sparsity of the brain-to-feature mapping. Our findings provide quantitative guidelines for diagnosing output dimension collapse and for designing effective translators and feature representations for zero-shot reconstruction.
URL: https://openreview.net/forum?id=RQiXUK4kQr
---
Title: From Mice to Trains: Amortized Bayesian Inference on Graph Data
Authors: Svenja Jedhoff, Elizaveta Semenova, Aura Raulo, Anne Meyer, Paul-Christian Bürkner
Abstract: Graphs arise across diverse domains, from biology and chemistry to social and information networks, as well as in transportation and logistics. Inference on graph-structured data requires methods that are permutation-invariant, scalable across varying sizes and sparsities, and capable of capturing complex long-range dependencies, making posterior estimation on graph parameters particularly challenging.
Amortized Bayesian Inference (ABI) is a simulation-based framework that employs generative neural networks to enable fast, likelihood-free posterior inference. We adapt ABI to graph data to address these challenges to perform inference on node-, edge-, and graph-level parameters. Our approach couples permutation-invariant graph encoders with flexible neural posterior estimators in a two-module pipeline: a summary network maps attributed graphs to fixed-length representations, and an inference network approximates the posterior over parameters. In this setting, several neural architectures can serve as the summary network. In this work we evaluate multiple architectures and assess their performance on controlled synthetic settings and two real-world domains - biology and logistics - in terms of recovery and calibration.
URL: https://openreview.net/forum?id=vpIeCm7YEA
---
Title: HyperAdapt: Simple High-Rank Adaptation
Authors: Abel Gurung, Joseph Campbell
Abstract: Foundation models excel across diverse tasks, but adapting them to specialized applications often requires fine-tuning, an approach that is memory and compute-intensive. Parameter-efficient fine-tuning (PEFT) methods mitigate this by updating only a small subset of weights. In this paper, we introduce HyperAdapt, a parameter-efficient fine-tuning method that significantly reduces the number of trainable parameters compared to state-of-the-art methods like LoRA. Specifically, HyperAdapt adapts a pre-trained weight matrix by applying row- and column-wise scaling through diagonal matrices, thereby inducing a high-rank update while requiring only $n+m$ trainable parameters for an $n \times m$ matrix. Theoretically, we establish an upper bound on the rank of HyperAdapt's updates, and empirically, we confirm that it consistently induces high-rank transformations across model layers. Experiments on GLUE, arithmetic reasoning, and commonsense reasoning benchmarks with models up to 14B parameters demonstrate that HyperAdapt matches or nearly matches the performance of full fine-tuning and state-of-the-art PEFT methods while using orders of magnitude fewer trainable parameters.
URL: https://openreview.net/forum?id=uhk13aXVxC
---
Title: ActionEQA: Action Interface for Embodied Question Answering
Authors: Tianwei Bao, Qineng Wang, Kangrui Wang, Mingkai Deng, Guangyi Liu, Jiayuan Mao, Lawrence Birnbaum, Zhiting Hu, Eric P. Xing, Zhaoran Wang, Manling Li
Abstract: While Vision-Language Models (VLMs) are increasingly integral to embodied intelligence, a significant action understanding bottleneck persists in translating high-level semantic instructions into precise low-level physical actions. However, current benchmarks for embodied agents primarily focus on high-level perception and planning, failing to capture the depth and nature of this semantic-to-physical gap. To address this, we introduce ActionEQA, the first Embodied Question Answering (EQA) benchmark designed to methodically evaluate the ability of VLMs to bridge this critical yet underexplored semantic-physical divide. Grounded in real-world robotics data, ActionEQA thoroughly analyzes VLMs’ grasp of the action interface using a dual-tier design: (1) a Three-Tiered Action Hierarchy for pinpointing the depth at which VLMs' action reasoning collapses. (2) Bidirectional Reasoning Tasks for testing whether VLMs struggle more to predict action outcomes or infer the actions that led to them. Our key findings reveal: (1) The primary bottleneck in action understanding occurs at the mid-level, arising from the challenge of grounding compositional language in 3D physical geometry. (2) VLMs are more adept at inferring past actions than predicting their future outcomes. (3) Richer visual inputs require greater spatial reasoning from VLMs to map actions to physical geometry. (4) Within the action hierarchy, model failures shift from predominantly perceptual errors at the high level to flawed geometric and physical reasoning at the low level.
URL: https://openreview.net/forum?id=HY2ruqdMt4
---
Title: OT Score: An OT based Confidence Score for Prototype-Assisted Source Free Unsupervised Domain Adaptation
Authors: Yiming Zhang, Sitong Liu, Alex Cloninger
Abstract: We address the computational and theoretical limitations of current distributional alignment methods for source-free unsupervised domain adaptation (SFUDA) using source class-mean features. In particular, we focus on estimating classification performance and confidence in the absence of target labels. Current theoretical frameworks for these methods often yield computationally intractable quantities and fail to adequately reflect the properties of the alignment algorithms employed. To overcome these challenges, we introduce the Optimal Transport (OT) score, a confidence metric derived from a novel theoretical analysis that exploits the flexibility of decision boundaries induced by Semi-Discrete Optimal Transport alignment. The proposed OT score is intuitively interpretable and theoretically rigorous. It provides principled uncertainty estimates for any given set of target pseudo-labels. Experimental results demonstrate that OT score outperforms existing confidence scores. Moreover, it improves SFUDA performance through training-time reweighting and provides a reliable, label-free proxy for model performance.
URL: https://openreview.net/forum?id=VQu8cWE9yJ
---
Title: Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learning
Authors: Matthew Lowery, John Turnage, Zachary Morrow, John Davis Jakeman, Akil Narayan, Shandian Zhe, Varun Shankar
Abstract: This paper introduces the Kernel Neural Operator (KNO), a provably convergent operator-learning architecture that utilizes compositions of deep kernel-based integral operators for function-space approximation of operators (maps from functions to functions). The KNO decouples the choice of kernel from the numerical integration scheme (quadrature), thereby naturally allowing for operator learning with explicitly-chosen trainable kernels on irregular geometries. On irregular domains, this allows the KNO to utilize domain-specific quadrature rules. To help ameliorate the curse of dimensionality, we also leverage an efficient dimension-wise factorization algorithm on regular domains. More importantly, the ability to explicitly specify kernels also allows the use of highly expressive, non-stationary, neural anisotropic kernels whose parameters are computed by training neural networks. We present universal approximation theorems showing that both the continuous and fully discretized KNO are universal approximators on operator learning problems. Numerical results demonstrate that on existing benchmarks the training and test accuracy of KNOs is comparable to or higher than popular operator learning techniques while typically using an order of magnitude fewer trainable parameters, with the more expressive kernels proving important to attaining high accuracy. KNOs thus facilitate low-memory, geometrically-flexible, deep operator learning, while retaining the implementation simplicity and transparency of traditional kernel methods from both scientific computing and machine learning.
URL: https://openreview.net/forum?id=Q0C4jYZQ7x
---
Title: AIMing for Standardised Explainability Evaluation in GNNs: A Framework and Case Study on Graph Kernel Networks
Authors: Magdalena Proszewska, Siddharth N
Abstract: Graph Neural Networks (GNNs) have advanced significantly in handling graph-structured data, but a comprehensive framework for evaluating explainability remains lacking. Existing evaluation frameworks primarily involve post-hoc explanations, and operate in the setting where multiple methods generate a suite of explanations for a single model. This makes comparison of explanations across models difficult. Evaluation of inherently interpretable models often targets a specific aspect of interpretability relevant to the model, but remains underdeveloped in terms of generating insight across a suite of measures. We introduce AIM, a comprehensive framework that addresses these limitations by measuring Accuracy, Instance-level explanations, and Model-level explanations. AIM is formulated with minimal constraints to enhance flexibility and facilitate broad applicability. Here, we use AIM in a pipeline, extracting explanations from inherently interpretable GNNs such as graph kernel networks (GKNs) and prototype networks (PNs), evaluating these explanations with AIM, identifying their limitations and obtaining insights to their characteristics. Taking GKNs as a case study, we show how the insights obtained from AIM can be used to develop an updated model, xGKN, that maintains high accuracy while demonstrating improved explainability. Our approach aims to advance the field of Explainable AI (XAI) for GNNs, providing more robust and practical solutions for understanding and improving complex models.
URL: https://openreview.net/forum?id=onZkYXI7oe
---
Title: Diffusion Models for Solving Inverse Problems via Posterior Sampling with Piecewise Guidance
Authors: Saeed Mohseni-Sehdeh, Walid Saad, Kei Sakaguchi, Tao Yu
Abstract: Diffusion models are powerful tools for sampling from high-dimensional distributions by progressively transforming pure noise into structured data through a denoising process. When equipped with a guidance mechanism, these models can also generate samples from conditional distributions. In this paper, a novel diffusion-based framework is introduced for solving inverse problems using a piecewise guidance scheme. The guidance term is defined as a piecewise function of the diffusion timestep, facilitating the use of different approximations during high-noise and low-noise phases. This design is shown to effectively balance computational efficiency with the accuracy of the guidance term.
Unlike task-specific approaches that require retraining for each problem, the proposed method is problem-agnostic and readily adaptable to a variety of inverse problems. Additionally, it explicitly incorporates measurement noise into the reconstruction process.
The effectiveness of the proposed framework is demonstrated through extensive experiments on image restoration tasks, specifically image inpainting and super-resolution.
Using a class conditional diffusion model for recovery, compared to the pseudoinverse-guided diffusion
model ($\Pi$GDM) baseline, the proposed framework achieves a reduction in inference time of $25\%$ for inpainting with both random and center masks, and $23\%$ and $24\%$ for $4\times$ and $8\times$ super-resolution tasks, respectively, while incurring only negligible loss in PSNR and SSIM.
URL: https://openreview.net/forum?id=nvw3XfvBi7
---
Title: Pull-to-Outlier & Contrastive Objective-level (POCO) Unlearning: A Framework for Sample and Objective Forgetting
Authors: Agil Aghasanli, Plamen P Angelov
Abstract: Current Machine Unlearning (MU) methods require full retraining or extensive fine-tuning, lack formal removal criteria, and focus only on sample-level forgetting, limiting their practicality. We address these gaps with two lightweight, projection-only techniques operating above frozen feature extractors. Pull-to-Outlier Unlearning (POU) offers a transparent, unsupervised geometric removal method by displacing embeddings of unwanted samples or entire classes into synthetic outlier regions, while preserving downstream performance and distilling knowledge of the remaining data. To the best of our knowledge, Contrastive Objective-level Unlearning (COU) is the first method to remove learned objectives. It perturbs projection weights to eliminate a target task’s influence. Then it realigns the original data manifold, which can provide the possibility for managing agentic learning behaviors. We validate POU on CIFAR10, CIFAR100, and Caltech-256 with ResNet-based backbones, showing efficient instance and class forgetting with minimal impact on retained accuracy. COU is tested on DINO and CLIP feature representations, demonstrating effective objective-level erasure while preserving all non-target tasks.
URL: https://openreview.net/forum?id=KQxEwiA0VE
---
Title: Hierarchical Filtering and Refinement Classification for Few-Shot Class-Incremental Learning
Authors: Li-Jun Zhao, Zhen-Duo Chen, Xin Luo, Xin-Shun Xu
Abstract: Few-shot class-incremental learning (FSCIL) aims at recognizing novel classes continually with limited novel class samples. A mainstream baseline for FSCIL is first to train the whole model in the base session, then freeze the feature extractor in the incremental sessions. Despite achieving high overall accuracy, most methods exhibit notably low accuracy on incremental classes. While some recent methods have recognized this issue, their strategies remain constrained by a unified classification objective across all samples, making it difficult to simultaneously satisfy the performance requirements of both base and incremental classes. In this paper, considering that base and incremental classes play different yet both critical roles in FSCIL, we approach FSCIL from a more structured perspective by decomposing the overall classification objective into three sub-objectives. Building on this insight, we propose a novel classification framework called Hierarchical Filtering and Refinement Classification (HFRC) to hierarchically decompose and address the classification task. Extensive experiments demonstrate that our method effectively balances the classification accuracy between base and incremental classes, and achieves superior performance compared to state-of-the-art methods.
URL: https://openreview.net/forum?id=7MXra1JSh8
---
Title: Language-Pretraining-Induced Bias: A Strong Foundation for General Vision Tasks
Authors: Yaxin Luo, Zhiqiang Shen
Abstract: The ratio of "outlier" parameters in language pre-training models and vision pre-training models differs significantly, making cross-modality (language and vision) inherently more challenging than cross-domain adaptation. As a result, many previous studies have focused on cross-domain transfer rather than attempting to bridge language and vision modalities, assuming that language pre-trained models are unsuitable for downstream visual tasks due to disparate parameter spaces. Contrary to this assumption, we show that adding a "bridge training" stage as a modality adaptation learner can effectively align Large Language Model (LLM) parameters with vision tasks. Specifically, we propose a simple yet powerful solution random label bridge training that requires no manual labeling and helps LLM parameters adapt to vision foundation tasks. Moreover, our findings reveal that partial bridge training is often advantageous, as certain layers in LLMs exhibit strong foundational properties that remain beneficial even without fine-tuning for visual tasks. This surprising discovery opens up new avenues for leveraging language pre-trained parameters directly within vision models and highlights the potential of partial bridge training as a practical pathway to cross-modality adaptation.
URL: https://openreview.net/forum?id=N7DSUbnzYo
---
Title: Diffusion Models are Secretly Zero-Shot 3DGS Harmonizers
Authors: Vsevolod Skorokhodov, Nikita Durasov, Pascal Fua
Abstract: Gaussian Splatting has become a popular technique for various 3D Computer Vision tasks, including novel view synthesis, scene reconstruction, and dynamic scene rendering. However, the challenge of natural-looking object insertion, where the object's appearance seamlessly matches the scene, remains unsolved. In this work, we propose a method, dubbed D3DR, for inserting a 3DGS-parametrized object into a 3DGS scene while correcting its lighting, shadows, and other visual artifacts to ensure consistency. We reveal a hidden ability of diffusion models trained on large real-world datasets to implicitly understand correct scene lighting, and leverage it in our pipeline. After inserting the object, we optimize a diffusion-based Delta Denoising Score (DDS)-inspired objective to adjust its 3D Gaussian parameters for proper lighting correction. We introduce a novel diffusion personalization technique that preserves object geometry and texture across diverse lighting conditions, and utilize it to achieve consistent identity matching between original and inserted objects. Finally, we demonstrate the effectiveness of the method by comparing it to existing approaches, achieving 2.0 dB PSNR improvements in relighting quality.
URL: https://openreview.net/forum?id=1jjIitxVmM
---
Title: ExpertLens: Activation steering features are highly interpretable
Authors: Masha Fedzechkina, Eleonora Gualdoni, Sinead Williamson, Katherine Metcalf, Skyler Seto, Barry-John Theobald
Abstract: Activation steering methods in large language models (LLMs) have emerged as an effective way to perform targeted updates to enhance generated language without requiring large amounts of adaptation data. We ask whether the features discovered by activation steering methods are interpretable. We identify neurons responsible for specific concepts (e.g., ''cat'') using the ''finding experts'' method from research on activation steering and show that the ExpertLens, i.e., inspection of these neurons, provides insights about model representation. We find that ExpertLens representations are stable across models and datasets and closely align with human representations inferred from behavioral data, matching inter-human alignment levels. ExpertLens significantly outperforms the alignment captured by word/sentence embeddings and sparse autoencoder (SAE) features. By reconstructing human concept organization through ExpertLens, we show that it enables a granular view of LLM concept representation. Our findings suggest that ExpertLens is a flexible and lightweight approach for capturing and analyzing model representations.
URL: https://openreview.net/forum?id=FBIsN6RdYO
---
Title: Constraint-Aware Flow Matching via Randomized Exploration
Authors: Zhengyan Huan, Jacob Boerma, Liping Liu, Shuchin Aeron
Abstract: We consider the problem of designing constraint-aware flow matching (FM) models that address the issue of constraint violations commonly observed in vanilla generative models. We consider two scenarios, viz.: (a) when a differentiable distance function to the constraint set is given, and (b) when the constraint set is only available via queries to a membership oracle. For case (a), we propose a simple adaptation of the FM objective with an additional term that penalizes the distance between the constraint set and the generated samples. For case (b), we propose to employ randomization and learn a mean flow that is numerically shown to have a high likelihood of satisfying the constraints. This approach deviates significantly from existing works that require simple convex constraints, knowledge of a barrier function, or a reflection mechanism to constrain the probability flow. Furthermore, in the proposed setting we show that a two-stage approach, where both stages approximate the same original flow but with only the second stage probing the constraints via randomization, is more computationally efficient than the corresponding one-stage approach. Through several synthetic cases of constrained generation, we numerically show that the proposed approaches achieve significant gains in terms of constraint satisfaction while matching the target distributions. As a showcase for a practical oracle-based constraint, we show how our approach can be used for training an adversarial example generator, using queries to a hard-label black-box classifier. We conclude with several future research directions. Our code is available at \url{https://github.com/ZhengyanHuan/FM-RE}.
URL: https://openreview.net/forum?id=OR4h9WPJhV
---
Title: Mixtures of Locally Bounded Langevin dynamics for Bayesian Model Averaging
Authors: Kilian Zepf, Tareen Dawood, Aasa Feragen, Ender Konukoglu
Abstract: Properties of probability distributions change when going from low to high dimensions, to the extent that they exhibit counterintuitive behavior. Gaussian distributions intuitively illustrate a well-known effect of moving to higher dimensions, namely that the typical set almost surely does not contain the mean, which is the distribution's most probable point. This can be problematic in Bayesian Deep Learning, as the samples drawn from the high-dimensional posterior distribution are often used as Monte Carlo samples to estimate the integral of the predictive distribution. Here, the predictive distribution will reflect the behavior of the samples and, therefore, of the typical set. For instance, we cannot expect to sample networks close to the maximum a posteriori estimate after fitting a Gaussian approximation to the posterior using the Laplace method. In this paper, we introduce a method that aims to mitigate this typicality problem in high dimensions by sampling from the posterior with Langevin dynamics on a restricted support enforced by a reflective boundary condition. We demonstrate how this leads to improved posterior estimates by illustrating its capacity for fine-grained out-of-distribution (OOD) ranking on the Morpho-MNIST dataset.
URL: https://openreview.net/forum?id=ibqfadKjgo
---
Title: Investigating Linguistic Steering: An Analysis of Adjectival Effects Across Large Language Model Architectures
Authors: Lars Malmqvist
Abstract: Achieving reliable control of Large Language Models (LLMs) requires a precise, scalable understanding of how they interpret linguistic cues. We introduce a rigorous framework using Shapley values to quantify the steering effect of individual adjectives on model performance, moving beyond anecdotal heuristics to principled attribution. Applying this method to 100 adjectives across a diverse suite of models (including o3, gpt-4o-mini, phi-3, llama-3-70b, and deepseek-r1) on the MMLU benchmark, we uncover several critical findings for AI alignment. First, we find that a small subset of adjectives act as disproportionately powerful "levers," yet their effects are not universal. Cross-model analysis reveals a "family effect": models of a shared lineage exhibit correlated sensitivity profiles, while architecturally distinct models react in a largely uncorrelated manner, challenging the notion of a one-size-fits-all prompting strategy. Second, focused follow-up studies demonstrate that the steering direction of these powerful adjectives is not intrinsic but is highly contingent on their syntactic role and position within the prompt. For larger models like gpt-4o-mini, we provide the first quantitative evidence of strong, non-additive interaction effects where adjectives can synergistically amplify, antagonistically dampen, or even reverse each other's impact. In contrast, smaller models like phi-3 exhibit a more literal and less compositional response. These results suggest that as models scale, their interpretation of prompts becomes more sophisticated but also less predictable, posing a significant challenge for robustly steering model behavior and highlighting the need for compositional and model-specific alignment techniques.
URL: https://openreview.net/forum?id=xN7NYpQeBm
---
Title: Adaptive Model Selection in Offline Contextual MDP's without Stationarity
Authors: Riddhiman Bhattacharya, Sayak Chakrabarty, Imon Banerjee
Abstract: Contextual MDP's are powerful tools with wide applicability in areas from biostatistics to machine learning. However, specializing them to offline datasets has been challenging due to a lack of robust, theoretically backed, methods. Our work tackles this problem by introducing a new approach towards adaptive estimation and cost optimization of contextual MDP's. This estimator, to the best of our knowledge, is the first of its kind, and is endowed with strong optimality guarantees. We achieve this by overcoming the key technical challenges evolving from the endogenous properties of contextual MDP's; such as non-stationarity, or model irregularity. Our guarantees are established under complete generality by utilizing the relatively recent and powerful statistical technique of $T$-estimation (Baraud, 2011). We first provide a procedure for selecting an estimator given a sample from a contextual MDP and use it derive oracle risk bounds under two distinct, but nevertheless meaningful, loss functions. We then consider the problem of determining the optimal control with the aid of the aforementioned density estimate and provide finite sample guarantees for the cost function.
URL: https://openreview.net/forum?id=FGBZ4q1HPZ
---
Title: SAMAT: A Stereotype-Aware Multimodal Transformer for Interpretable Misogynistic Meme Detection
Authors: gopendra Vikram singh, Arpan Phukan, Kushal Kanwar, Asif Ekbal
Abstract: This paper introduces SAMAT, a Stereotype-Aware Multimodal Alignment Transformer for detecting and explaining implicit misogyny in memes, where harm arises from subtle visual-textual incongruity and cultural stereotypes. SAMAT integrates three components: a Stereotype Subspace Projection Module (SSPM) that structures representations; a fidelity-based retrieval mechanism aligned with a curated Rationale Bank; and an evidence-conditioned explanation generator. For evaluation, we rely on the MEE corpus with 8,000 explanations, Stereotype Alignment (SAS) and Contextual Faithfulness (CFS) scores. Experiments show that SAMAT achieves a Macro-F1 of 88.1%, surpassing MLLM baselines, while improving retrieval faithfulness (SAS: 0.78) and explanation grounding (CFS: 0.68). Ablations confirm gains stem from structured stereotype projection and evidential retrieval, not scale. SAMAT offers a transparent, culturally grounded framework for accountable content moderation, aligning with Responsible AI objectives.
URL: https://openreview.net/forum?id=1DJBFS7rLW
---
Title: Tumor-anchored deep feature random forests for out-of-distribution detection in lung cancer segmentation
Authors: Aneesh Rangnekar, Harini Veeraraghavan
Abstract: Accurate segmentation of lung tumors from 3D computed tomography (CT) scans is essential for automated treatment planning and response assessment. Despite self-supervised pretraining on numerous datasets, state-of-the-art transformer backbones remain susceptible to out-of-distribution (OOD) inputs, often producing confidently incorrect segmentations with potential risk in clinical deployment. Hence, we introduce RF-Deep, a lightweight post-hoc random forests-based framework that uses deep features trained with limited outlier exposure, requiring as few as 40 labeled scans (20 in-distribution and 20 OOD), to improve scan-level OOD detection. RF-Deep repurposes the hierarchical features from the pretrained-then-finetuned segmentation backbones, aggregating features from multiple regions-of-interest anchored to predicted tumor regions to capture OOD likelihood.
We evaluated RF-Deep on 2,232 CT volumes spanning near-OOD (pulmonary embolism, COVID-19 negative) and far-OOD (kidney cancer, healthy pancreas) datasets. RF-Deep achieved AUROC $>$~93 on the challenging near-OOD datasets, where it outperformed the next best method by 4--7 percentage points, and produced near-perfect detection (AUROC $>$~99) on far-OOD datasets. The approach also showed transferability to two blinded validation datasets under the ensemble configuration (COVID-19 positive and breast cancer; AUROC $>$~94). RF-Deep maintained consistent performance across backbones of different depths and pretraining strategies, demonstrating applicability of post-hoc detectors as a safety filter for clinical deployment of tumor segmentation pipelines.
URL: https://openreview.net/forum?id=XmjYlBxFxn
---
Title: Learning-Augmented Robust Algorithmic Recourse
Authors: Kshitij Kayastha, Vasilis Gkatzelis, Shahin Jabbari
Abstract: Algorithmic recourse provides individuals who receive undesirable outcomes from machine learning systems with minimum-cost improvements to achieve a desirable outcome. However, machine learning models often get updated, so the recourse may not lead to the desired outcome. The robust recourse framework chooses recourses that are less sensitive to adversarial model changes, but this comes at a higher cost. To address this, we initiate the study of learning-augmented algorithmic recourse and evaluate the extent to which a designer equipped with a prediction of the future model can reduce the cost of recourse when the prediction is accurate (consistency) while also limiting the cost even when the prediction is inaccurate (robustness). We propose a novel algorithm, study the robustness-consistency trade-off, and analyze how prediction accuracy affects performance.
URL: https://openreview.net/forum?id=IFssttzxnP
---
Title: On Symmetric Losses for Policy Optimization with Noisy Preferences
Authors: Soichiro Nishimori, Yu-Jie Zhang, Thanawat Lodkaew, Masashi Sugiyama
Abstract: Optimizing policies based on human preferences is key to aligning language models with human intent.
This work focuses on reward modeling, a core component in reinforcement learning from human feedback (RLHF), and offline preference optimization, such as direct preference optimization.
Conventional approaches typically assume accurate annotations. However, real-world preference data often contains noise due to human errors or biases, which can be asymmetric.
We propose a principled framework for robust policy optimization under noisy preferences based on the view of reward modeling as a binary classification problem.
Specifically, we demonstrate that asymmetric preference noise can be effectively treated as symmetric noise under this framework.
This viewpoint allows us to leverage symmetric losses, well known for their robustness to label noise in classification, for reward modeling, which leads to our Symmetric Preference Optimization (SymPO) method, a novel offline preference optimization algorithm.
Theoretically, we prove that symmetric losses enable successful policy improvement even with noisy labels, as the resulting reward is rank-preserving—a property we identify as sufficient for policy improvement.
Empirical evaluations on a synthetic dataset and real-world language model alignment tasks demonstrate that SymPO achieves competitive or higher performance than existing robust methods in high-noise scenarios.
URL: https://openreview.net/forum?id=cBWGLmSeao
---
Title: Multivariate Conformal Prediction using Optimal Transport
Authors: Michal Klein, Louis Béthune, Eugene Ndiaye, marco cuturi
Abstract: Conformal prediction (CP) provides distribution-free uncertainty quantification by constructing prediction sets whose validity relies on ranking conformity scores. Because ranking requires an ordering, most CP methods use univariate scores; extending them to multivariate settings, where no canonical order for vectors exists, remains challenging. We build on the theory of Monge--Kantorovich quantiles and ranks to propose a geometry-aware scalarization of vector-valued scores: we transport multivariate conformity scores to the spherical uniform distribution on the unit ball via an entropic optimal transport (OT) map and use the transported radius as a scalar score. Standard split conformal calibration then applies directly, preserving finite-sample marginal coverage. The resulting method, OTCP, produces prediction regions that adapt to the empirical geometry of the score distribution, going beyond the ellipsoidal sets imposed by norm-based scalarizations. Across a benchmark of 24 multivariate regression datasets, OTCP improves efficiency and conditional-coverage metrics mainly in low output dimensions ($d \leq 4$), while we also study the computational and statistical trade-offs involved in estimating entropic OT maps.
URL: https://openreview.net/forum?id=LrXAq63eT7
---
Title: Quantile $Q$-Learning: Revisiting Offline Extreme $Q$-Learning with Quantile Regression
Authors: Xinming Gao, Shangzhe Li, Yujin Cai, Wenwu Yu
Abstract: Offline reinforcement learning (RL) enables policy learning from fixed datasets without further environment interaction, making it particularly valuable in high-risk or costly domains. Extreme $Q$-Learning (XQL) is a recent offline RL method that models Bellman errors using the Extreme Value Theorem, yielding strong empirical performance. However, XQL and its stabilized variant MXQL suffer from notable limitations: both require extensive hyperparameter tuning specific to each dataset and domain, and also exhibit instability during training. To address these issues, we proposed a principled method to estimate the temperature coefficient $\beta$ via quantile regression under mild assumptions. To further improve training stability, we introduce a value regularization technique with mild generalization, inspired by recent advances in constrained value learning. Experimental results demonstrate that the proposed algorithm achieves competitive or superior performance across a range of benchmark tasks, including D4RL and NeoRL2, while maintaining stable training dynamics and using a consistent set of hyperparameters across all datasets and domains.
URL: https://openreview.net/forum?id=tBKznsUimN
---
Title: Riemannian Generative Decoder
Authors: Andreas Bjerregaard, Søren Hauberg, Anders Krogh
Abstract: Euclidean representations distort data with intrinsic non-Euclidean structure. While Riemannian representation learning offers a solution by embedding data onto matching manifolds, it typically relies on an encoder to estimate densities on chosen manifolds. This involves optimizing numerically brittle objectives, potentially harming model training and quality. To completely circumvent this issue, we introduce the Riemannian generative decoder, a unifying approach for finding manifold-valued latents on any Riemannian manifold. Latents are learned with a Riemannian optimizer while jointly training a decoder network. By discarding the encoder, we vastly simplify the manifold constraint compared to current approaches which often only handle few specific manifolds. We validate our approach on three case studies --- a synthetic branching diffusion process, human migrations inferred from mitochondrial DNA, and cells undergoing a cell division cycle --- each showing that learned representations respect the prescribed geometry and capture intrinsic non-Euclidean structure. Our method requires only a decoder, is compatible with existing architectures, and yields interpretable latent spaces aligned with data geometry.
URL: https://openreview.net/forum?id=vuPMXg1FDT
---
Title: Optimal Regret and Hard Violation for Constrained Markov Decision Processes with Adversarial Losses and Constraints
Authors: Srinjoy Roy, Swagatam Das
Abstract: We investigate online learning in finite-horizon episodic Constrained Markov Decision Processes (CMDPs) under the most demanding setting: adversarial losses and constraints, bandit feedback, and unknown transitions. The most popular approaches, such as primal-dual or linear programming, either rely on Slater's condition (which can yield vacuous bounds) or require solving a complex optimization problem at each round. Inspired by the groundbreaking work of~\citet{sinha2024optimal} in Constrained Online Convex Optimization (COCO), we map the CMDP instances to a corresponding COCO problem, thus creating simple and elegant algorithms that require only a single Euclidean projection per episode. Our algorithm first attains $\mathcal{\widetilde{O}}(\sqrt{T})$ regret and $\mathcal{\widetilde{O}}(\sqrt{T})$ hard cumulative constraint violation for adversarial losses and constraints, unknown transition dynamics, bandit feedback, without Slater's condition and also without access to a strictly feasible policy. We achieve $\mathcal{O}(\sqrt{T})$ regret and $\mathcal{\widetilde{O}}(\sqrt{T})$ hard violation for known transitions. Additionally, we study the remaining three permutations of known-unknown transitions and full-bandit feedback, again achieving optimal regret and hard violation bounds in each case. Besides closing several gaps in the literature, our simple construction of biased estimators for the sub-gradient could be of independent interest for didactic purposes. Finally, we conducted rigorous experiments on several CMDP instances to verify our theoretical results from a practical perspective.
URL: https://openreview.net/forum?id=EsInBaX0ko
---
Title: Model Debiasing by Learnable Data Augmentation
Authors: Pietro Morerio, Ruggero Ragonesi, Vittorio Murino
Abstract: Deep Neural Networks are well known for efficiently fitting training data, yet experiencing poor generalization capabilities whenever some kind of bias dominates over the actual task labels, resulting in models learning “shortcuts”. In essence, such models are often prone to learn spurious correlations between data and labels. In this work, we tackle the problem of learning from biased data in the very realistic unsupervised scenario, i.e., when the bias is unknown. This is a much harder task as compared to the supervised case, where auxiliary, bias-related annotations, can be exploited in the learning process. This paper proposes a novel 2-stage learning pipeline featuring a data augmentation strategy able to regularize the training. First, biased/unbiased samples are identified by training over-biased models.
Second, such subdivision (typically noisy) is exploited within a data augmentation framework, properly combining the original samples while learning mixing parameters, which has a regularization effect. Experiments on synthetic and realistic biased datasets show state-of-the-art classification accuracy, outperforming competing methods, ultimately proving robust performance on both biased and unbiased examples. Notably, being our training method totally agnostic to the level of bias, it also positively affects performance for any, even apparently unbiased, dataset, thus improving the model generalization regardless of the level of bias (or its absence) in the data.
URL: https://openreview.net/forum?id=3ac7heNftC
---
Title: Learning Multimodal Energy-Based Model with Multimodal Variational Auto-Encoder via MCMC Revision
Authors: Jiali Cui, Zhiqiang Lao, Heather Yu
Abstract: Energy-based models (EBMs) are a flexible class of deep generative models and are well-suited to capture complex dependencies in multimodal data. However, learning multimodal EBM by maximum likelihood requires Markov Chain Monte Carlo (MCMC) sampling in the joint data space, where noise-initialized Langevin dynamics often mixes poorly and fails to discover coherent inter-modal relationships. Multimodal VAEs have made progress in capturing such inter-modal dependencies by introducing a shared latent generator and a joint inference model. However, both the shared latent generator and joint inference model are parameterized as unimodal Gaussian (or Laplace), which severely limits their ability to approximate the complex structure induced by multimodal data. In this work, we study the learning problem of the multimodal EBM, shared latent generator, and joint inference model. We present a learning framework that effectively interweaves their MLE updates with corresponding MCMC refinements in both the data and latent spaces. Specifically, the generator is learned to produce coherent multimodal samples that serve as strong initial states for EBM sampling, while the inference model is learned to provide informative latent initializations for generator posterior sampling. Together, these two models serve as complementary models that enable effective EBM sampling and learning, yielding realistic and coherent multimodal EBM samples. Extensive experiments demonstrate superior performance for multimodal synthesis quality and coherence compared to various baselines. We conduct various analyses and ablation studies to validate the effectiveness and scalability of the proposed multimodal framework.
URL: https://openreview.net/forum?id=ZVD7bHNpY1
---
Title: The Shape of Attraction in UMAP: Exploring the Embedding Forces in Dimensionality Reduction
Authors: Mohammad Tariqul Islam, Jason W. Fleischer
Abstract: Uniform manifold approximation and projection (UMAP) is among the most popular neighbor embedding methods. The method samples pairs of point indices according to similarities in the high-dimensional space, and applies attractive and repulsive forces to their coordinates in the low-dimensional embedding. In this paper, we analyze the forces to reveal their effects on cluster formations and visualization, and compare UMAP to its contemporaries. Repulsion emphasizes differences, controlling cluster boundaries and inter-cluster distance. Attraction is more subtle, as attractive tension between points can manifest simultaneously as attraction and repulsion in the lower-dimensional mapping. This explains the need for learning rate annealing and motivates the different treatments between attractive and repulsive terms. Moreover, by modifying attraction, we improve the consistency of cluster formation under random initialization. Overall, our analysis provides a mechanistic understanding of UMAP and related embedding methods.
URL: https://openreview.net/forum?id=fdPNhqav5G
---
New submissions
===============
Title: Compositional Neuro-Symbolic Reasoning
Abstract: We study structured abstraction-based reasoning for the Abstraction and Reasoning Corpus (ARC) and compare its generalization to test-time approaches. Purely neural architectures lack reliable combinatorial generalization, while strictly symbolic systems struggle with perceptual grounding. We therefore propose a neuro-symbolic architecture that extracts object-level structure from grids, uses neural priors to propose candidate transformations from a fixed domain-specific language (DSL) of atomic patterns, and filters hypotheses using cross-example consistency. Instantiated as a compositional reasoning framework based on unit patterns inspired by human visual abstraction, the system augments large language models (LLMs) with object representations and transformation proposals. On ARC-AGI-2, it improves base LLM performance from 16% to 24.4% on the public evaluation set, and to 30.8% when combined with ARC Lang Solver via a meta-classifier. These results demonstrate that separating perception, neural-guided transformation proposal, and symbolic consistency filtering improves generalization without task-specific finetuning or reinforcement learning, while reducing reliance on brute-force search and sampling-based test-time scaling.
URL: https://openreview.net/forum?id=Sa0cXYcJp0
---
Title: Better Models, Faster Training: Sigmoid Attention for single-cell Foundation Models
Abstract: Training stable biological foundation models requires rethinking attention mechanisms: we find that using sigmoid attention as a drop in replacement for softmax attention a) produces better learned representations: on six diverse single-cell datasets, sigmoid achieves 25% higher cell-type separation, better cell-type cohesion metrics, and lower validation loss, b) faster training, models with sigmoid attention train up to 10% faster than their softmax counterparts, and c) more stable training by eliminating inherent sources of instability in softmax attention. We establish that sigmoid attention has globally bounded derivatives ($\leq 0.25$) as opposed to softmax, and a diagonal Jacobian structure in contrast with softmax's dense coupling, which together help alleviate training instabilities. In stress tests on 160M-parameter bidirectional attention models trained without gradient clipping on 8K-token sequences, softmax diverges catastrophically, with gradients exploding by four orders of magnitude, while sigmoid remains stable. Finally, we implement and open-source TritonSigmoid, an efficient GPU kernel that achieves 515 TFLOPS on H100 GPUs, outperforming both FlashAttention-2 and FlashSigmoid, with native padding support, which is essential for biological sequences. Our results establish sigmoid attention as both theoretically grounded and empirically superior for biological foundation models. Code will be made available upon publication.
URL: https://openreview.net/forum?id=GxmQsP5CcU
---
Title: Learning as Homeostasis: Beyond the Optimization Paradigm in Machine Intelligence
Abstract: The dominant paradigm in machine intelligence defines learning as the minimisation of an empirical loss function. While successful, this approach often results in brittle systems prone to catastrophic forgetting and reward hacking, contrasting with the homeostatic stability observed in biological organisms. We propose Constraint‑First Machine Learning (CFML), a paradigm that redefines learning as the maintenance of feasibility under an expanding set of structural constraints rather than the pursuit of a global optimum. Utilising Viability Projection Updates (VPU) based on stochastic differential inclusions, we demonstrate that learning can occur as a stochastic drift within a viability manifold; for data‑driven constraints, only a tiny Constraint Anchor Set (e.g., five exemplars per class) is stored to detect violations, with no full replay required. Our evaluations on ResNet‑18 architectures demonstrate that CFML sustains 85.9% final accuracy and 93.3% relative retention across sequential tasks, whereas Elastic Weight Consolidation retains only 42% and naive SGD collapses to chance level. Even experience replay, with the same memory budget, gradually erodes safety boundaries. Furthermore, we show that CFML resolves localised conflicts where structural invariants directly contradict empirical data—a scenario where standard optimisation either violates safety or suffers significant utility stagnation, while Lagrangian regularisation leads to complete utility collapse. By shifting the objective of machine intelligence from minimisation to viability, CFML provides a mathematically rigorous framework for safe, stable, and biologically plausible lifelong learning.
URL: https://openreview.net/forum?id=sgvPGWZ1uv
---
Title: Operational Feature Fingerprints of Graph Datasets via a White-Box Signal-Subspace Probe
Abstract: Graph neural networks achieve strong node-classification performance, but learned message
passing entangles ego features, neighborhood smoothing, high-pass graph differences, class
geometry, and classifier-boundary effects inside opaque representations. This makes it difficult
to determine why nodes are classified as they are, and which graph-learning mechanisms
are useful, harmful, or necessary for a given dataset. We propose WG-SRC (White-box
Graph Signal–Subspace Residual Classifier), a white-box signal-subspace probe for prediction
and graph dataset diagnosis. WG-SRC replaces learned message passing with an explicit,
named graph-signal dictionary containing raw features, row- and symmetric-normalized
low-pass propagation, and high-pass graph differences. It then combines Fisher coordinate
selection, class-wise PCA subspaces, closed-form multi-α ridge classification, and validation-
based score fusion. Because every signal block and decision module is explicit, the fitted
scaffold produces both predictions and an operational fingerprint over raw-feature, low-pass,
high-pass, class-geometric, and ridge-boundary mechanisms. Across six node-classification
datasets, WG-SRC remains competitive with aligned reproduced baselines and achieves a
positive average gain under matched repeated splits. Its fingerprints distinguish low-pass-
dominated Amazon graphs, mixed high-pass and class-geometrically complex Chameleon
behavior, and raw- or boundary-sensitive WebKB graphs. Aligned interventions further
show that these fingerprints are operational: they identify when high-pass blocks behave
like removable noise, when graph-derived or raw signals should be preserved, and when
ridge-type boundary correction matters. Additional fixed black-box component probes further
show that measured dataset fingerprints organize architectural behavior across multiple
black-box families: different measured dataset conditions repeatedly favor different inductive
biases. Thus, WG-SRC serves both as a functioning white-box classifier and as a dataset-
fingerprinting probe, enabling fingerprint-conditioned analysis of how black-box graph-model
components behave under different measured dataset conditions.
URL: https://openreview.net/forum?id=ZkzpSrZzUv
---
Title: On-Policy Model Error Suffices: An Invariant-Measure Return-Gap Bound for Model-Based Reinforcement Learning
Abstract: We study the discounted return gap between a fixed policy evaluated on a true dynamical system and on a learned closed-loop model. Lipschitz-based bounds in the model-based reinforcement learning literature control this gap by the \emph{supremum} of the one-step model error over the state space, amplified by the global closed-loop Lipschitz constant; this is pessimistic for systems whose closed-loop trajectories concentrate on a low-dimensional attractor. We prove a return-gap bound whose dominant term is the one-step model error \emph{averaged under the invariant measure of the true closed loop}, amplified by a trajectory-localized linearised contraction rate, plus geometrically-decaying transients. The bound recovers the classical sup-norm bound as a limiting case; its leading term is strictly smaller whenever the invariant-measure-averaged error \(\bar\eps_\mu\) is strictly below the global supremum error \(\eps_0\), as occurs when large model errors lie off the closed-loop attractor. We exhibit a regime in which this distinction is qualitative: the classical bound is infinite while ours is finite. As a consequence, the empirical on-policy mean-squared error minimized by modern world-model algorithms upper-bounds (up to a square-root and a finite-sample concentration term) the return-gap-controlling quantity, giving the training
objective an explicit return-gap interpretation. We extend the result to stochastic dynamics via a Wasserstein-\(1\) coupling, and prove a matching bound on the Wasserstein distance between the true and learned-model invariant measures.
URL: https://openreview.net/forum?id=qDJ8axLhMa
---
Title: Score-based Membership Inference on Diffusion Models
Abstract: Membership inference attacks (MIAs) against Diffusion Models (DMs) raise pressing privacy concerns by revealing whether a sample was part of the training set. While existing methods typically rely on measuring reconstruction error across multiple denoising steps as a test statistic, they often incur significant computational overhead. In this work, we present a simple yet successful attack statistic using only the predicted noise vectors from the DM's denoiser, or equivalently, the score. Specifically, we show that the expected denoiser output points toward a kernel-weighted local mean of nearby training samples, such that its norm encodes proximity to the training set and thereby reveals membership. Building on this observation, we propose SimA, a single-query attack that provides a principled, efficient alternative to existing multi-query methods. SimA consistently achieves superior performance across variants of DMs and the Latent Diffusion Models (LDMs) on eight different datasets. Its Monte Carlo variant (SimA-MC) exhibits state-of-the-art performance across all experiments, significantly outperforming baseline methods in terms of TPR@1%FPR. These results demonstrate that complex reconstruction trajectories are unnecessary for effective membership inference, establishing SimA as a highly efficient benchmark for auditing privacy in DMs and LDMs.
URL: https://openreview.net/forum?id=Ckvsu5xRmf
---
Title: ANU-RL: A New Perspective on Unsupervised Representation Learning for Visual Place Recognition
Abstract: Representation Learning (RL) is fundamental for image matching, retrieval, classification, and other applications, enabling task-specific feature learning. RL algorithms aim to learn compact embeddings that preserve the neighbourhood structure of the input data. A general approach to this is contrastive learning, which pulls similar images (positives) closer together and pushes dissimilar images (negatives) farther apart in the embedding space. In Visual Place Recognition (VPR), positive images of a query share specific geographical and visual attributes with the query and can, form a cluster. In contrast, negative images differ from the query and may vary among themselves or be similar. % may vary or be similar among themselves. Most existing training objectives focus only on the relationships between query-positives and query-negatives. In this work, we hypothesize that, in addition to these relationships, other naturally available relationships, such as positives-to-negatives and intra-positives, can improve VPR performance by enhancing representation quality. The proposed framework, A New Perspective on Unsupervised Representation Learning (ANU-RL), when integrated with VPR aggregators like BoQ, SALAD, MixVPR, and NetVLAD, achieves state-of-the-art performance on most challenging VPR benchmarks, including Pittsburgh 30k, Tokyo 24/7, Nordland, MSLS (val), and many others. Moreover, all of this comes at no extra cost at inference time. Further, we generalize the proposed framework to a wider range of metric learning applications, specifically image retrieval.
URL: https://openreview.net/forum?id=mXE4OP55il
---
Title: Verify What Matters: Budgeted Verification for Tool Using Agents under Counterfactual Downstream Harm
Abstract: Tool-using agents make intermediate decisions that alter persistent state, shape later observations,
and can trigger failures that are not equally easy to recover from. When verification
is costly, the central question is therefore not whether checking helps in general, but which
decisions are worth checking. Policies driven only by local uncertainty capture whether a step
may be wrong, but not how much that error would matter if left uncorrected. We formulate
budgeted verification for tool-using agents as an intervention-allocation problem, and argue
that the step-level value of verification factors into verifier efficacy, local error probability,
counterfactual downstream harm, and intervention cost. This yields a consequence-aware
decision target under which uncertainty-only routing is the restriction to a constant harm.
We emphasize that the paper’s primary contribution is conceptual: a decision structure
in which local error likelihood and downstream harm enter as separate inputs rather than
being collapsed into a single scalar score. Empirically, we report a four-episode mechanism
pilot on a dependency-sensitive slice of an OpenClaw-based sandbox, where uncertainty-only
routing misses two irreversible failures that a harm-aware rule catches. The pilot is intentionally
small-scale; it isolates the mechanism rather than estimating an effect size, and a
Fisher exact test on the pilot’s 2×2 contingency yields p ≈ 0.43. We specify three follow-up
comparisons, including matched-budget, cross-slice, and process-reward-threshold, whose results
would be required for an effect-size claim; these are outlined but not reported in the present
submission. Readers should interpret the empirical content as supporting the framework’s
direction of effect in a restricted setting, not as establishing quantitative superiority.
URL: https://openreview.net/forum?id=nv1jzr0FaZ
---
Title: Let’s Grow an Unbiased Community : Guiding the Fairness of Graphs via New Links
Abstract: Graph Neural Networks (GNNs) have achieved remarkable success across diverse applications. However, due to the biases in the graph structures, graph neural networks face significant challenges in fairness. Although the original user graph structure is generally biased, it is promising to guide these existing structures toward unbiased ones by introducing new links. The fairness guidance via new links could foster unbiased communities, thereby enhancing fairness in downstream applications. To address this issue, we propose a novel framework named FairGuide. Specifically, to ensure fairness in downstream tasks trained on fairness-guided graphs, we introduce a differentiable community detection task as a pseudo downstream task. Our theoretical analysis further demonstrates that optimizing fairness within this pseudo task effectively enhances structural fairness, promoting fairness generalization across diverse downstream applications. Moreover, FairGuide employs an effective strategy which leverages meta-gradients derived from the fairness-guidance objective to identify new links that significantly enhance structural fairness. Extensive experimental results demonstrate the effectiveness and generalizability of our proposed method across a variety of graph-based fairness tasks.The code and datasets are available at https://anonymous.4open.science/r/FairGuide-8907/README.md.
URL: https://openreview.net/forum?id=7JyRR9aQX2
---
Title: Physiology-Informed Diffusion for 12-Lead ECG Generation
Abstract: Large-scale 12-lead ECG data are critical for training reliable cardiac machine learning
systems, yet their availability is limited by privacy constraints, annotation cost, and severe
class imbalance. Generative models offer a promising solution, but standard diffusion models
typically treat ECGs as generic multivariate time series and do not explicitly exploit known
physiological structure.
We propose PhysDiff-ECG, a physiology-guided diffusion framework that integrates cardiac
ordinary differential equation (ODE) prior into the diffusion trajectory. Our central idea
is to make ECG physiology tractable during training by deriving differentiable regularizers
from a dynamical model of cardiac activity together with a differentiable 12-lead observation
model. Given a denoised reconstruction along the reverse process, PhysDiff-ECG fits a latent
physiological explanation via an unrolled inner optimization and penalizes violations of both
the simulator dynamics and the induced ECG reconstruction.
This training-time regularization biases the learned denoising trajectories toward physio-
logically realizable ECGs while preserving the flexibility of latent diffusion. Experiments
on standard 12-lead ECG benchmarks show that PhysDiff-ECG improves physiological fi-
delity, representation-space realism, class-conditional diagnostic consistency, and downstream
classification performance relative to strong GAN and diffusion baselines.
URL: https://openreview.net/forum?id=Q0fgUVvNkb
---
Title: Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation
Abstract: Dataset condensation constructs compact synthetic datasets that retain the training utility of large real-world datasets, enabling efficient model development and potentially supporting downstream research in governed domains such as healthcare. Trajectory matching (TM) is a widely used condensation approach that supervises synthetic data using changes in model parameters observed during training on real data, yet the structure of this supervision signal remains poorly understood. In this paper, we provide a geometric characterisation of trajectory matching, showing that a fixed synthetic dataset can only reproduce a limited span of such training-induced parameter changes. When the resulting supervision signal is spectrally broad, this creates a conditional representability bottleneck. Motivated by this mismatch, we propose Bézier Trajectory Matching (BTM), which replaces SGD trajectories with quadratic Bézier trajectory surrogates between initial and final model states. These surrogates are optimised to reduce average loss along the path while replacing broad SGD-derived supervision with a more structured, lower-rank signal that is better aligned with the optimisation constraints of a fixed synthetic dataset, and they substantially reduce trajectory storage. Experiments on five clinical datasets demonstrate that BTM consistently matches or improves upon standard trajectory matching, with the largest gains in low-prevalence and low-synthetic-budget settings. These results indicate that effective trajectory matching depends on structuring the supervision signal rather than reproducing stochastic optimisation paths.
URL: https://openreview.net/forum?id=9qTlh7f0z3
---
Title: On the Analysis of the One-to-Many Mapping in Cross-Modality Text-to-Video Generation with Semantic Spaces
Abstract: Despite recent advances in text-to-video generation, the role of text and video latent spaces in learning a semantically shared representation remains underexplored. In this cross-modality generation task, most methods rely on conditioning the video generation process by injecting the text representation into it, rather than exploring the implicit shared knowledge between the modalities. However, the feature-based alignment of both modalities is not straightforward, especially for the one-to-many mapping scenario in which one text can be mapped to several valid semantically aligned videos, a challenge that generally produces a representation collapse in the alignment phase. In this work, we investigate and give insights into how both modalities cope in a shared semantic space where each modality representation is previously learned in an unsupervised way. We explore this from a latent space learning perspective with a plug-and-play framework that adopts autoencoder-based models that could be used with other representations. We show that the one-to-many case requires different alignment strategies than those commonly used in the literature, which struggle to align both modalities in a semantically shared space.
URL: https://openreview.net/forum?id=Khl7xjzTak
---
Title: Inducing Artificial Uncertainty in Language Models
Abstract: In safety-critical applications, language models should be able to characterize their uncertainty with meaningful probabilities. Many uncertainty quantification approaches require supervised data; however, finding suitable unseen challenging data is increasingly difficult for large language models trained on vast amounts of scraped data. If the model is consistently (and correctly) confident in its predictions, the uncertainty quantification method may consistently overestimate confidence on new and unfamiliar data. Finding data which exhibits enough uncertainty to train supervised uncertainty quantification methods for high-performance models may therefore be challenging, and will increase in difficulty as LLMs saturate datasets. To address this issue, we first introduce the problem of inducing artificial uncertainty in language models, then investigate methods of inducing artificial uncertainty on trivially easy data in the absence of challenging data at training time. We use probes trained to recognize artificial uncertainty on the original model, and find that these probes trained on artificial uncertainty outperform probes trained without artificial uncertainty in recognizing real uncertainty, achieving notably higher calibration on hard data with minimal loss of performance on easy data.
URL: https://openreview.net/forum?id=cUcWgu0wGU
---
Title: On Memory: A Comparison of Memory Mechanisms in World Models
Abstract: World models enable agents to plan within imagined environments by predicting future states conditioned on past observations and actions. However, their ability to plan over long horizons is limited by the effective memory span of the backbone architecture. This limitation leads to perceptual drift in long rollouts, degrading the model's capacity to recall recently observed scenes. In this work, we investigate the effective memory span of transformer-based world models through an analysis of memory augmentation mechanisms. We introduce a taxonomy that distinguishes between memory \emph{encoding} and memory \emph{injection} mechanisms, motivating their roles in extending the world model's memory through the lens of residual stream dynamics. We evaluate twenty combinations of four encoding methods and five injection methods in the MemoryMaze environment. Using a state recall evaluation task across multiple imagination horizons, we measure the memory recall capacity of each mechanism and analyze their respective trade-offs in reconstruction quality, latent prediction error, and computational cost. We further ablate the effect of injection depth and compare the best memory-augmented vision transformer against a pure state-space model backbone. Our central finding is that the mLSTM memory encoder outperforms all alternatives in both reconstruction and latent fidelity metrics. Paired with additive injection, it exhibits the strongest recall capabilities at a moderate computational cost while matching or slightly exceeding a pure Mamba backbone.
URL: https://openreview.net/forum?id=21XSQF0dmM
---
Title: Bridging the High-Frequency Data Gap: A Millisecond-Resolution Network Dataset for Advancing Time Series Foundation Models
Abstract: Time series foundation models (TSFMs) require diverse, real-world datasets to adapt across varying domains and temporal frequencies. However, current large-scale datasets predominantly focus on low-frequency time series with sampling intervals, i.e., time resolution, in the range of seconds to years, hindering their ability to capture the nuances of high-frequency time series data. To address this limitation, we introduce a novel dataset that captures millisecond-resolution wireless and traffic conditions from an operational 5G wireless deployment, expanding the scope of TSFMs to incorporate high-frequency data for pre-training. Further, the dataset introduces a new domain, wireless networks, thus complementing existing more general domains like energy and finance. The dataset also provides use cases for short-term forecasting, with prediction horizons spanning from 1 millisecond (1 step) to 96 milliseconds (96 steps). By benchmarking traditional machine learning models and TSFMs on predictive tasks using this dataset, we demonstrate that most TSFM model configurations perform poorly on this new data distribution in both zero-shot and fine-tuned settings. Our work underscores the importance of incorporating high-frequency datasets during pre-training and forecasting to enhance architectures, fine-tuning strategies, generalization, and robustness of TSFMs in real-world applications.
URL: https://openreview.net/forum?id=dMlzHYA13f
---
Title: Rethinking Reward Models for Multi-Domain Test-Time Scaling
Abstract: The reliability of large language models (LLMs) during test-time scaling is often assessed with external verifiers or reward models that distinguish correct reasoning from flawed logic. Prior work generally assumes that process reward models (PRMs), which score every intermediate reasoning step, outperform outcome reward models (ORMs) that assess only the final answer. This view is based mainly on evidence from narrow, math-adjacent domains. We present the first unified evaluation of four reward model variants, discriminative ORM and PRM (dORM, dPRM) and generative ORM and PRM (gORM, gPRM), across 14 diverse domains. Contrary to conventional wisdom, we find that (i) dORM performs on par with dPRM, (ii) gPRM is not competitive, and (iii) overall, gORM is the most robust, yielding significant and consistent gains across every tested domain. We attribute this to PRM-style stepwise scoring, which inherits label noise from LLM auto-labeling and has difficulty evaluating long reasoning trajectories, including those involving self-correcting reasoning. Our theoretical analysis shows that step-wise aggregation compounds errors as reasoning length grows, and our empirical observations confirm this effect. These findings challenge the prevailing assumption that fine-grained supervision is always better and support generative outcome verification for multi-domain deployment. We publicly release our code at thttps://anonymous.4open.science/r/TMLR-Multi-RM-7BD5 to facilitate future research in multi-domain settings.
URL: https://openreview.net/forum?id=PgouBhL7IR
---
Title: Can Large Reasoning Models Self-Train?
Abstract: Recent successes of reinforcement learning (RL) in training large reasoning models motivate the question of whether self-training, the process where a model learns from its own judgments, can be sustained within RL. In this work, we study this question using majority voting as a simple self-feedback mechanism. On a comprehensive set of experiments on both synthetic and real reasoning tasks, we find that this basic approach improves not only the model's reasoning performance, but also its capability of generating better quality feedback for the next RL iteration, driving further model improvement. Yet our analysis also reveals a critical limitation of such a self-training paradigm: prolonged RL with self-reward leads to reward hacking where models learn to maximize training (pseudo-)reward, resulting in sudden performance collapse. Together, these results highlight feedback design as the central challenge and call for future research on mechanisms to enable prolonged self-improvement.
URL: https://openreview.net/forum?id=7GNKZjWU1U
---
Title: Sinkhorn doubly stochastic attention rank decay analysis
Abstract: The self-attention mechanism is central to the success of Transformer architectures. However, standard row-stochastic attention has been shown to suffer from significant signal degradation across layers. In particular, it can induce rank collapse, resulting in increasingly uniform token representations, as well as entropy collapse, characterized by highly concentrated attention distributions. Recent work has highlighted the benefits of doubly stochastic attention as a form of entropy regularization, promoting a more balanced attention distribution and leading to improved empirical performance. In this paper, we study rank collapse across network depth and show that doubly stochastic attention matrices normalized with Sinkhorn algorithm preserve rank more effectively than standard softmax row-stochastic ones. As previously shown for softmax, skip connections are crucial to mitigate rank collapse. We empirically validate this phenomenon on both sentiment analysis and image classification tasks. Moreover, we derive a theoretical bound for the pure self-attention rank decay when using Sinkhorn normalization and find that rank decays to one doubly exponentially with depth, a phenomenon that has already been shown for softmax.
URL: https://openreview.net/forum?id=fGItYoS8j1
---
Title: RoZO: Geometry-Aware Zeroth-Order Fine-Tuning on the Low-Rank Adapters for Black-Box Large Language Models
Abstract: Large language models (LLMs) have achieved remarkable success across a wide range of tasks, yet fine-tuning them efficiently under black-box or memory-constrained settings remains challenging. Parameter-efficient fine-tuning (PEFT) techniques such as LoRA alleviate memory usage by restricting updates to low-rank adapters, while zeroth-order (ZO) optimization further avoids back-propagation by estimating gradients from function evaluations. Recent work, such as LOZO, leverages random low-rank perturbations to reduce the variance of ZO estimates, but it overlooks the intrinsic geometric structure of LoRA adapters and suffers from unstable convergence and limited integration with adaptive optimizers. To address these limitations, we propose \textbf{RoZO}, a Riemannian zeroth-order optimization framework that constrains updates to the tangent space of the LoRA manifold. By exploiting geometry-aware updates with parallel transport, adaptive preconditioning, and trust-region control, RoZO achieves more stable convergence, tighter variance bounds, and superior performance compared to existing ZO methods.
URL: https://openreview.net/forum?id=BClq0UuYzL
---
Title: Communication-Efficient Federated Low-Rank Update Algorithm and its Connection to Implicit Regularization
Abstract: Federated Learning (FL) faces significant challenges related to communication efficiency and performance reduction when scaling to many clients. To address these issues, we explore the potential of using low-rank updates and provide the first theoretical study of rank properties in FL. Our theoretical analysis shows that a client’s loss exhibits a higher-rank structure (i.e., gradients span higher-rank subspaces of the Hessian) compared to the server’s loss, and that low-rank approximations of the clients’ gradients have greater similarity. Based on this insight, we hypothesize that constraining client-side optimization to a low-rank subspace could provide an implicit regularization effect while reducing communication costs. Consequently, we propose FedLoRU, a general low-rank update framework for FL. Our framework enforces low-rank client-side updates and accumulates these updates to form a higher-rank model. We are able to establish convergence of the algorithm; the convergence rate matches FedAvg. Experimental results demonstrate that FedLoRU performs comparably to full-rank algorithms and exhibits robustness to heterogeneous and large numbers of clients.
URL: https://openreview.net/forum?id=MXSAMrmktX
---
Title: LungTTA: Text-to-Audio Generation of Synthetic Lung Sounds for Respiratory Health
Abstract: Respiratory audio analysis is still limited by data scarcity, as real recordings are difficult to collect and often involve privacy and clinical constraints, which makes it harder to train robust machine learning models. We introduce LungTTA, a text-to-audio framework based on a latent diffusion model, which generates respiratory sounds such as cough, breathing, and phonation from structured prompts. The model is fine-tuned on 116,660 publicly available recordings and includes a retrieval-based memory component together with watermarking for traceability. We evaluate the generated audio using Fréchet Audio Distance (FAD), Kullback–Leibler (KL) divergence, and Inception Score (IS), and also introduce PRISM (Pulmonary Respiratory Integrity & Similarity Metric) a domain aware metric designed to capture respiratory signal structure. LungTTA achieves a FAD of 2.72, KL of 0.50, IS of 1.22, and PRISM of 0.23, compared to Stable Audio Open (6.73, 0.67) for FAD and KL, Make-An-Audio (1.54) for IS, and RespAgent (0.24) for PRISM. In human evaluation, LungTTA achieves 80.91 (Overall Quality, OVL) and 75.13 (Relevance to Text, REL), compared to RespAgent (59.27, 58.97) and EZAudio (55.24, 52.69), while expert assessment yields 58.33 (OVL), 44.44 (REL), and 38.89 (Clinical Relevance for Assessment, CRA), compared to RespAgent (56.94, 43.06, 36.11) and EZAudio (36.11, 29.17, 33.33). In a downstream COVID-19 cough classification task, LungTTA improves performance under a VGGish-based setting, increasing AUC from 0.7331 (no augmentation) and 0.7631 (classical augmentation) to 0.7701 using LungTTA. These results demonstrate that LungTTA-generated synthetic respiratory audio can be used as an effective data augmentation method.
URL: https://openreview.net/forum?id=oRZWkKtCls
---
Title: Analyzing the Effect of Noise in LLM Fine-tuning
Abstract: Fine-tuning is the dominant paradigm for adapting pretrained large language models (LLMs) to downstream NLP tasks. In practice, fine-tuning datasets may contain various forms of noise arising from annotation errors, preprocessing artifacts, or automated data collection. While prior work has focused on designing robust learning algorithms to mitigate performance degradation under noisy conditions, comparatively little is known about how different types of noise affect the internal learning dynamics of LLMs during fine-tuning. In this work, we systematically study the impact of noise on model behavior across three pretrained model families (GPT-2, Qwen2 and Llama-2) and three diverse NLP tasks. We introduce controlled perturbations corresponding to three common real-world noise types: label noise, grammatical noise, and typographical noise. Beyond task-level performance, we analyze layer-wise representation changes and attention patterns to understand how noise propagates through the network. Our results show that corrupting labels (i.e. label noise) consistently causes the largest performance degradation, whereas grammatical noise and typographical noise can occasionally yield mild regularization benefits. We further find that noise effects are localized primarily to task-specific layers, while attention structures remain comparatively stable.
URL: https://openreview.net/forum?id=NlSBeHZEz5
---
Title: Reinforcement Learning for LLM Post-Training: A Survey
Abstract: Large language models (LLMs) trained via pretraining and supervised fine-tuning (SFT) can still produce harmful and misaligned outputs, or struggle in domains like math and coding. Reinforcement learning (RL)-based post-training methods, including Reinforcement Learning from Human Feedback (RLHF) methods like Direct Preference Optimization (DPO) and Reinforcement Learning with Verifiable Rewards (RLVR) approaches like PPO and GRPO, have made remarkable gains to alleviate these issues. Yet, no existing work offers a technically detailed comparison of the various methods driving this progress.
In order to fill this gap, we present a timely survey that connects foundational components with latest advancements. We derive a single policy gradient framework that unifies pretraining, SFT, RLHF, and RLVR as special cases while also organizing the more recent techniques therein. The main contributions of our survey are as follows: (1) a self-contained introduction to MLE, RLHF, and RLVR foundations and the unified policy gradient framework; (2) detailed technical analysis of PPO- and GRPO-based methods alongside offline and iterative DPO approaches, decomposed along prompt sampling, response sampling, and gradient coefficient axes; (3) standardized notation enabling direct cross-method comparison; and (4) comprehensive comparison of implementation details and empirical results of each method in the appendix. We aim to serve as a technically grounded reference for researchers and practitioners working on LLM post-training.
URL: https://openreview.net/forum?id=UdsXTNzzvg
---
Title: Improving the Usefulness of Decision Trees as Explanations
Abstract: In classification with tabular data, one often utilizes tree-based models. Those can be competitive with deep neural networks on tabular data and, under some conditions, explainable. The explainability depends on the tree’s depth and the accuracy of each leaf. Decision trees containing leaves with unbalanced accuracy can provide misleading explanations. Low-accuracy leaves provide less useful explanations to the individuals they classify. Here, we train a shallow tree with the objective of minimizing the maximum misclassification error across each leaf node. The shallow tree provides a more useful global explanation, while its overall statistical performance can become comparable to that of state-of-the-art methods by extending the leaves with additional models.
URL: https://openreview.net/forum?id=y4FJDMpWq3
---
Title: Monte Carlo Multi-Feature Baseline Shapley (MMBS): An axiomatic attribution method for fine-grained explanations of image classification networks
Abstract: This paper presents the Multi-Feature Baseline Shapley (MBS) attribution method for explaining the outcome of a neural network for a given input. MBS generalizes the Integrated Gradients (IG) and Baseline Shapley (BShap) methods by introducing a step size parameter. When the step size is set to one, MBS equals BShap, and when it is set to the number of features, MBS equals IG. MBS is an axiomatic method, which means that it was designed to satisfy certain axioms (mathematical properties). These axioms ensure that the attribution maps relate to the neural network in appealing ways, for example, by preserving linearity or symmetry. We prove that MBS satisfies eight axioms that are also satisfied by IG and BShap. To quickly approximate MBS, this paper presents the Monte Carlo Multi-Feature Baseline Shapley (MMBS) method, which is an unbiased estimator of MBS. On image classification tasks, we show that MMBS also approximates a Monte Carlo estimate for BShap while being up to 20,000 times faster to compute. Furthermore, we compare MMBS to nine configurations of existing attribution methods on three image classification networks trained on either the Fashion MNIST or ImageNet1k dataset. MMBS has the best area under the deletion curve score on all three networks.
URL: https://openreview.net/forum?id=LLFIcr7zWh
---
Title: The Impact of Enforcing Representational Consistency of Identical Transformations for Disentangled Representation
Abstract: Recent symmetry-based approaches in Variational Autoencoders (VAEs) have advanced disentanglement learning and compositional generalization. However, existing methods can encode identical semantic transformations differently depending on the specific sample pairs, which reduce the representational consistency of identical transformations. In this paper, we analyze how three commonly used symmetry parameterization families in prior work, namely (1) matrix-exponential parameterizations over the general linear group GL(n), (2) vector-additive actions in latent space, and (3) surjective mappings from latent vectors to the unit circle, can make it difficult to represent identical transformations consistently in dimension-wise disentangled latent spaces. To address this issue, we propose a framework that maps latent vectors to a bijective cyclic representation on the unit circle via the Cayley transform, together with a fixed-grid codebook regularization. We study this problem in a controlled setting and develop practical weakly supervised and supervised variants. Experiments on disentanglement benchmarks and compositional generalization tasks show that the proposed framework yields improved disentanglement performance and strong compositional generalization under supervised settings, with the stronger-supervision variants providing empirical reference points for the representational capacity of the framework. Overall, our results suggest that consistent representation of identical transformations is a useful design principle for improving disentanglement and generalization performance in the considered setting.
URL: https://openreview.net/forum?id=VjbBxj4aWb
---
Title: Foundation Models in Robotics: A Comprehensive Review of Methods, Models, Datasets, Challenges and Future Research Directions
Abstract: Over the recent years, the field of robotics has been undergoing a transformative paradigm shift from fixed, single-task, domain-specific solutions towards adaptive, multi-function, general-purpose agents, capable of operating in complex, open-world, dynamic environments. This tremendous advancement is primarily driven by the emergence of Foundation Models (FMs), i.e., large-scale neural-network architectures trained on massive, internet-scale, heterogeneous datasets that provide unprecedented capabilities in multi-modal understanding/reasoning, long-horizon planning, and cross-embodiment generalization. In this context, the current study provides a holistic, thorough, systematic, and in-depth review of the research landscape of FMs in robotics. In particular, the evolution in the field is initially delineated through five distinct research phases, spanning from the early incorporation of native Natural Language Processing (NLP) and Computer Vision (CV) models to the current frontier of multi-sensory generalization and real-world deployment. Subsequently, a highly-granular, multi-criteria, taxonomic investigation of the literature methods is performed, examining the following key aspects: a) The employed foundation model types (i.e., LLMs, VFMs, VLMs, and VLAs), b) The underlying neural network architectures, c) The adopted learning paradigms, d) The different learning stages of knowledge incorporation, e) The most common robotic tasks (including perception, planning, navigation, manipulation, and human-robot interaction), and f) The main real-world application domains. For each defined criterion/aspect, a methodical comparative analysis of the various categories of approaches and critical insights are provided. Moreover, a thorough report on the publicly available datasets, required for model training and evaluation, is provided per considered robotic task. Furthermore, a comprehensive and hierarchical discussion on the current open challenges and promising future research directions in the field is incorporated.
URL: https://openreview.net/forum?id=qF7vdPrpPk
---
Title: Confidence as Control: A Survey of Confidence Utilization in Large Language Models
Abstract: Most work on confidence in large language models has focused on estimation, uncertainty quantification, and calibration. In deployed systems, however, the key question is how confidence should be used to govern behavior. This survey studies $\textbf{confidence utilization}$: the use of confidence-related signals to control system decisions. We formalize this perspective through a unified framework in which confidence is defined over decision units under a local state and then consumed by a policy to determine actions. Using this lens, we organize the literature across full LLM lifecycle: training, inference, model selection and cascading, retrieval-augmented generation, risk management, and agentic control. We compare methods by signal source, decision unit, and functional role, and conclude by highlighting open challenges in confidence semantics, composition, source attribution, decision-aware evaluation, and robustness. Overall, the survey positions confidence not only as an estimation target, but as a control primitive for building more reliable and trustworthy LLM systems.
URL: https://openreview.net/forum?id=jOYTx34FAY
---
Title: Structuring Semantic Embeddings for Principle Evaluation: A Kernel-Guided Contrastive Learning Approach
Abstract: Reliable post-hoc principle evaluation—verifying whether generated text adheres to predefined human values such as safety, fairness, or helpfulness—is a critical bottleneck in AI alignment. While general-purpose text embeddings are widely deployed for this task, they inherently struggle with fine-grained principle distinctions due to severe feature entanglement. Texts sharing similar vocabulary but representing diametrically opposed principles often collapse into the same representation space, blurring critical decision boundaries. To overcome this limitation without the prohibitive costs of full-parameter fine-tuning, we introduce Kernel-Guided Contrastive Learning (KGCL), a framework that transitions the evaluation paradigm from generic semantic approximation to explicit decision boundary sculpting. Operating as a lightweight module atop frozen generalist encoders, KGCL projects entangled embeddings into a structured, principle-aligned subspace. We mathematically prove that our composite objective establishes a defined geometric margin and establishes strict bounds on geometric clustering metrics. Extensive experiments validate these theoretical guarantees, demonstrating that KGCL dramatically enhances the linear separability of highly confusable classes and provides a geometric shield against majority collapse. Remarkably, our explicitly optimized embeddings not only achieve absolute F1 improvements of up to 19.4\% over task-agnostic contrastive baselines but also consistently outperform the implicit in-context reasoning of massive generative Large Language Models. Ultimately, KGCL establishes that targeted geometric sculpting provides a highly discriminative, computationally efficient paradigm for robust principle alignment.
URL: https://openreview.net/forum?id=nZCPG2Aarv
---
Title: How can deep transformers represent hierarchical languages? An expressivity analysis via bounded-depth grammars.
Abstract: Deep neural networks are widely believed to derive their expressive power from their ability to form hierarchical representations, capturing progressively more abstract and compositional features across layers. In language modeling, transformers have emerged as the dominant architecture, with early layers capturing local syntactic patterns and later layers encoding more complex clause-level dependencies. While this intuition has shaped model design, there remains a lack of rigorous theoretical work demonstrating how deep transformers represent such hierarchical structures. In this work, we analyze the expressiveness of deep transformer models through the formal lens of bounded-depth, non-recursive context-free grammars. For this class of grammars, we explicitly construct transformers with positional attention whose depth grows linearly with grammar depth, while the neuron count scales with the number of derivation-tree shapes and quadratically with the number of production rules. Our theoretical results support the linear representation hypothesis by demonstrating that these architectures possess the structural capacity to encode abstract grammatical states into low-dimensional, linearly separable subspaces within the residual stream.
URL: https://openreview.net/forum?id=7L3Ogck8mw
---
Title: Algorithmic Exclusions, Collective Inclusions: Surveying Algorithmic Harms and Collective Action in LGBTQIA2S+ and Marginalized Communities
Abstract: LGBTQIA2S+ and marginalized communities often face harms when interacting with algorithmic systems, such as misgendering, content suppression, and other forms of exclusion. In this paper, we examine the social context that enables these harms in LGBTQIA2S+ spaces, summarize the existing literature on algorithmic harms, and explore how communities can leverage collective action to regain their agency. We survey methods that exploit properties of machine learning systems, such as data dependence, adversarial vulnerability to resist these harms through collective action. We categorize existing approaches to resisting these harms, organized by four collective motivations: reporting and contesting harms (Model Auditing and Challenging Algorithmic Decision Strategies), opting out of model training or decision-making (Algorithmic Opt-Out Strategies), actively intervening to shift model behavior (Collective Intervention Strategies), and seeking recommendations for favorable outcomes (Decision Modification and Recommendation Strategies). Through a mapping review, we systematically chart where LGBTQIA2S+ and other marginalized communities appear in this literature and where they are absent. Our mapping reveals that while these communities are well-represented in platform-based strategies such as folk theorization and data activism, they are nearly absent from model-based methods such as adversarial techniques and algorithmic collective action, where machine learning researchers have focused their efforts. These gaps highlight opportunities for ML researchers and developers to build community-focused tools and methods that enable collectives to coordinate responses to algorithmic harms and regain agency over the systems that affect them. By mapping resistance methods across data-based, model-based, and platform-based mechanisms, we identify where the current literature falls short in supporting the communities most affected by algorithmic harms.
URL: https://openreview.net/forum?id=LnT8zUUerK
---
Title: Environmental Footprint of GenAI Research: Insights from the Moshi Foundation Model
Abstract: New multi-modal large language models (MLLMs) are continuously being trained and deployed, following rapid development cycles. This generative AI frenzy is driving steady increases in energy consumption, greenhouse gas emissions, and a plethora of other environmental impacts linked to datacenter construction and hardware manufacturing. Mitigating the environmental consequences of GenAI remains challenging due to an overall lack of transparency by the main actors in the field. Even when the environmental impacts of specific models are mentioned, they are typically restricted to the carbon footprint of the final training run, omitting the research and development stages.
In this work, we explore the impact of GenAI research through a fine-grained analysis of the compute spent to create Moshi, a 7B-parameter speech-text foundation model for real-time dialogue developed by Kyutai, a leading privately funded open science AI lab. For the first time, our study dives into the anatomy of compute-intensive MLLM research, quantifying the GPU-time invested in specific model components and training phases, as well as early experimental stages, failed training runs, debugging, and ablation studies.
Additionally, we assess the environmental impacts of creating Moshi from beginning to end using a life cycle assessment methodology: we quantify energy and water consumption, greenhouse gas emissions, and mineral resource depletion associated with the production and use of datacenter hardware.
Our detailed analysis allows us to provide actionable guidelines to reduce compute usage and environmental impacts of MLLM research, paving the way for more sustainable AI research.
URL: https://openreview.net/forum?id=uurX0xsr8G
---
Title: Creating a Causally Grounded Rating Method for Assessing the Robustness of AI Models for Time-Series Forecasting
Abstract: AI models, including both time-series-specific and general-purpose Foundation Models (FMs), have demonstrated strong potential in time-series forecasting across sectors like finance. However, these models are highly sensitive to input perturbations, which can lead to prediction errors and undermine trust among stakeholders, including investors and analysts. To address this challenge, we propose a causally grounded rating framework to systematically evaluate model robustness by analyzing statistical and confounding biases under various noisy and erroneous input scenarios. Our framework is applied to a large-scale experimental setup involving stock price data from six companies and evaluates both uni-modal and multi-modal models, including Vision Transformer-based (ViT) models and FMs. We introduce six types of input perturbations and twelve data distributions to assess model performance. Results indicate that multi-modal and time-series-specific FMs demonstrate greater robustness and accuracy compared to general-purpose models. Further, to validate our framework's usability, we conduct a user study showcasing time-series models’ prediction errors along with our computed ratings. The study confirms that our ratings reduce the difficulty for users in comparing the robustness of different models. Our findings can help stakeholders understand model behaviors in terms of robustness and accuracy for better decision-making even without access to the model weights and training data, i.e., black-box settings.
URL: https://openreview.net/forum?id=RMM9HvWptL
---
Title: Make Still Further Progress: Chain of Thoughts for Tabular Data Leaderboard
Abstract: Tabular data, a fundamental data format in machine learning, is predominantly utilized in competitions and real-world applications. The performance of tabular models---such as gradient boosted decision trees and neural networks---can vary significantly across datasets due to differences in feature distributions and task characteristics. Achieving top performance on each dataset often requires specialized expert knowledge. To address this variability, practitioners often aggregate the predictions of multiple models. However, conventional aggregation strategies typically rely on static combination rules and lack instance-level adaptability. In this work, we propose an in-context ensemble framework for tabular prediction that leverages large language models (LLMs) to perform dynamic, instance-specific integration of external model predictions. Without access to raw tabular features or semantic information, our method constructs a context around each test instance using its nearest neighbors and the predictions from a pool of external models. Within this enriched context, we introduce Chain of Tabular Thoughts (CoT$^2$), a prompting strategy that guides LLMs through multi-step, interpretable reasoning, making still further progress toward expert-level decision-making. Experimental results show that our method outperforms well-tuned baselines and standard ensemble techniques across a wide range of tabular datasets.
URL: https://openreview.net/forum?id=ZpjaYA6Wst
---
Title: TimePre: Bridging Accuracy, Efficiency, and Stability in Probabilistic Time-Series Forecasting
Abstract: We proposeTimePre, a simple framework that unifies the efficiency of Multilayer Perceptron (MLP)-based models with the distributional flexibility of Multiple Choice Learning (MCL) for Probabilistic Time-Series Forecasting (PTSF). Stabilized Instance Normalization (SIN), the core of TimePre, is a normalization layer that explicitly addresses the trade-off among accuracy, efficiency, and stability. SIN stabilizes the hybrid architecture by correcting channel-wise statistical shifts, thereby resolving the catastrophic hypothesis collapse. Extensive experiments on six benchmark datasets demonstrate that TimePre achieves the best Distortion on all six benchmarks and the best or highly competitive CRPS-Sum on most benchmarks. Critically, TimePre achieves inference speeds that are orders of magnitude faster than sampling-based models, and is more stable than prior MCL approaches.
URL: https://openreview.net/forum?id=yQLnvkJMbP
---
Title: Light4D: Training-Free Extreme Viewpoint 4D Video Relighting
Abstract: Recent advances in diffusion-based generative models have established a new paradigm for image and video relighting. However, extending these capabilities to 4D relighting remains challenging, due primarily to the scarcity of paired 4D relighting training data and the difficulty of maintaining temporal consistency across extreme viewpoints. In this work, we propose \textbf{\textit{Light4D}}, a novel training-free framework designed to synthesize consistent 4D videos under target illumination, even under extreme viewpoint changes. First, we introduce Disentangled Flow Guidance, a time-aware strategy that effectively injects lighting control into the latent space while preserving geometric integrity. Second, to reinforce temporal consistency, we develop Temporal Consistent Attention within the IC-Light architecture and further incorporate deterministic regularization to eliminate appearance flickering. Extensive experiments demonstrate that our method achieves competitive performance in temporal consistency and lighting fidelity, robustly handling camera rotations from $-90^{\circ}$ to $90^{\circ}$.
URL: https://openreview.net/forum?id=m788NZ8hJf
---
Title: Density-Scaled Regularization for Offline Reinforcement Learning
Abstract: Value-based offline RL methods are prone to overestimate the values of out-of-distribution (OOD) actions, and this is often addressed by regularizing the action-value function in the Bellman update. However, existing regularization methods can suffer from being too conservative, which can arise from over-penalizing the values for both in-distribution actions and out-of-support actions. We present a new regularization method for offline value-based methods, called Density-Scaled (DS) regularization, which penalizes the value function based on the relative action density of the behavior policy. We show a theoretical connection between our method and the existing Supported Value Regularization (SVR) method, demonstrating how the SVR solution for policy evaluation can be viewed as a limiting case of the solution from the DS regularized problem. Empirical results demonstrate that the DS penalty is competitive with the state-of-the-art techniques, more robust to misestimation of the behavior density compared to SVR, and allows greater flexibility in learning hyperparameters associated with the behavior policy.
URL: https://openreview.net/forum?id=nDPrzkQTj1
---
Title: A Robust $\widetilde{\mathcal{O}}(1/\sqrt{T})$ Rate for Unprojected TD Learning with Linear Function Approximation
Abstract: We investigate the finite-time convergence properties of Temporal Difference (TD) learning with linear function approximation, a cornerstone of reinforcement learning. We are interested in the so-called ``robust'' setting, where the convergence guarantee does not depend on the potential function's minimal curvature. While prior work has established convergence guarantees in this setting, these results typically rely on the artificial assumption that each iterate is projected onto a bounded set. Removing such a condition was left as an open problem by Bhandari et al. (COLT'18), hypothesizing the need for additional ``regularity conditions''. In this paper, we show that the simple unprojected TD(0) converges with a rate of $\widetilde{\mathcal{O}}\left(\frac{||\boldsymbol{\theta}^*||^2_2}{\sqrt{T}}\right)$ in expectation, even in the presence of Markovian noise. We do not require an additional regularity condition, but only a minor polylog correction to the learning rate. Our analysis reveals a novel self-bounding property of the TD updates and exploits it to guarantee bounded iterates.
URL: https://openreview.net/forum?id=Tj1B5WDyt8
---
Title: MINDFeed: Mutual Information-Guided Single-Network Consistency Learning for Semi-Supervised 3D Medical Image Segmentation
Abstract: Medical image segmentation models based on deep learning require dense voxel-level annotations, which are costly to obtain for 3D medical imaging tasks. To address this limitation, we propose MINDFeed (Mutual Information per Decoder as Feedback), a semi-supervised training pipeline for 3D medical image segmentation. MINDFeed estimates predictive uncertainty via mutual information across stochastic forward passes and uses this signal to adaptively modulate decoder representations as a feedback gate, encouraging consistency in reliable regions while suppressing ambiguous responses. Unlike many prior approaches, MINDFeed does not rely on student–teacher architectures, exponential moving averages, or multiple model instances, thereby maintaining architectural simplicity and training efficiency. We conduct extensive experiments on CT and MRI datasets, covering binary and multi-class segmentation tasks with both single- and multi-modal inputs, and demonstrate that MINDFeed consistently outperforms recent state-of-the-art semi-supervised methods. In addition to improved segmentation performance, MINDFeed exhibits reduced variability among test samples, highlighting its robustness under limited annotation settings.
URL: https://openreview.net/forum?id=YI7AX6mrZO
---
Title: ViP$^2$-CLIP: Visual-Perception Prompting with Unified Alignment for Zero-Shot Anomaly Detection
Abstract: Zero-Shot Anomaly Detection (ZSAD) aims to detect anomalies in a target dataset without any training samples, leveraging models trained on auxiliary data. While CLIP offers strong cross-modal representations for ZSAD, its pretraining objective inherently emphasizes global foreground semantics over fine-grained local defects. Consequently, its anomaly localization remains highly sensitive to prompt wording, severely limiting the effectiveness of existing methods that rely on explicit category labels. To overcome this limitation, we introduce ViP$^{2}$-CLIP, a lightweight CLIP-based ZSAD framework featuring Visual-Perception Prompting (ViP-Prompt) and Unified Text-Patch Alignment (UTPA). ViP-Prompt replaces fixed class-name tokens with image-conditioned cues to adaptively generate fine-grained prompts, obviating the need for manual templates and class-name priors. Furthermore, UTPA enforces a unified text-patch alignment strategy across multiple visual scales, jointly optimizing image-level detection and pixel-level localization. These mechanisms enable the model to precisely localize abnormal regions, exhibiting particular robustness in scenarios with ambiguous or privacy-constrained category labels. Extensive experiments on 14 industrial and medical benchmarks show that ViP$^{2}$-CLIP achieves superior performance over existing state-of-the-art approaches. Code is available at: https://anonymous.4open.science/r/Anonymous-11FF/.
URL: https://openreview.net/forum?id=KCRRuiQSIm
---
Title: Interpreting Temporal Graph Neural Networks with Koopman Theory
Abstract: Spatiotemporal graph neural networks (STGNNs) have shown promising results in many domains, from forecasting to epidemiology. However, understanding the dynamics learned by these models and explaining their behaviour is significantly more difficult than for models that deal with static data. Inspired by Koopman theory, which allows a simple description of intricate, nonlinear dynamical systems, we introduce new explainability approaches for temporal graphs. Specifically, we present two methods to interpret the STGNN's decision process and identify the most relevant spatial and temporal patterns in the input for the task at hand. The first relies on dynamic mode decomposition (DMD), a Koopman-inspired dimensionality reduction method. The second relies on sparse identification of nonlinear
dynamics (SINDy), a popular method for discovering governing equations of dynamical systems, which we use for the first time as a general tool for explainability. On semi-synthetic dissemination datasets, our methods correctly identify interpretable features such as the times at which infections occur and the infected nodes. We also validate the methods qualitatively on a real-world human motion dataset, where the explanations highlight the body parts most relevant for action recognition.
URL: https://openreview.net/forum?id=CKHBWc7wsa
---
Title: BiScale-GTR: Fragment-Aware graph Transformers for Multi-Scale Molecular Representation Learning
Abstract: Graph Transformers have recently attracted attention for molecular property prediction by combining the inductive biases of graph neural networks (GNNs) with the global receptive field of Transformers. However, many existing hybrid architectures remain GNN-dominated, causing the resulting representations to remain heavily shaped by local message passing. Moreover, most existing methods operate at only a single structural granularity, limiting their ability to capture molecular patterns that span multiple molecular scales. We introduce BiScale-GTR, a unified framework for self-supervised molecular representation learning that combines chemically grounded fragment tokenization with adaptive multi-scale reasoning. Our method improves graph Byte Pair Encoding (BPE) tokenization to produce consistent, chemically valid, and high-coverage fragment tokens, which are used as fragment-level inputs to a parallel GNN–Transformer architecture. Architecturally, atom-level representations learned by a GNN are pooled into fragment-level embeddings and fused with fragment token embeddings before Transformer reasoning, enabling the model to jointly capture local chemical environments, substructure-level motifs, and long-range molecular dependencies. Experiments on MoleculeNet, PharmaBench, and the Long Range
Graph Benchmark (LRGB) demonstrate state-of-the-art performance across both classification and regression tasks. Attribution analysis further shows that BiScale-GTR highlights chemically meaningful functional motifs, providing interpretable links between molecular structure and predicted properties. Code will be released upon acceptance.
URL: https://openreview.net/forum?id=97L9IRPWu7
---
Title: Spatial Correlation Structure Determines the Effectiveness of Channel Mixing Strategies in Time Series Forecasting
Abstract: Channel-dependent (CD) and channel-independent (CI) strategies represent competing inductive biases in long-term time series forecasting. While empirical studies suggest that CD strategies become more effective as channel correlation increases, the specific data characteristics that determine this have not been systematically quantified. We introduce two dataset properties to characterize the effectiveness of CI, CD, and hybrid models: the high-correlation fraction, defined as the proportion of highly correlated channel pairs, and block separation, defined as the degree of separation between channel clusters. Using the hybrid Series-cOre Fused Time Series (SOFTS) model as a controlled testbed, we develop a fully CD variant, Channel Mixer SOFTS (C-SOFTS), that maximizes channel interactions in both the spatial and frequency domains, and a fully CI variant, Identity SOFTS (I-SOFTS), that removes all channel interactions. We find that I-SOFTS consistently outperforms the hybrid on few-channel, low-correlation datasets. C-SOFTS outperforms the hybrid on datasets with high block separation, or with high-correlation fraction and a few clusters, achieving up to 15.9\% average MSE improvement. The hybrid proves optimal only when the high-correlation fraction and block separation are moderately low. These results show that the CI-CD choice is not a universal architectural decision but a dataset-dependent one. We advocate for reporting spatial dataset characteristics alongside performance metrics as a standard practice, enabling practitioners to match inductive biases to data regimes rather than relying on universal architectural rankings.
URL: https://openreview.net/forum?id=AmJiOQu9qh
---
Title: Observer-Side Diagnosis of Prompt-Induced Interference in Large Language Models: A Macro-Group Vocabulary and Targeted Cross-Lingual Stress Tests
Abstract: Small prompt fragments can change not only an intended surface property of a response (e.g., style), but also secondary reliability-relevant behaviors such as epistemic commitment, scope of alternatives, and reasoning presentation. Such interaction-level behavioral shifts are difficult to characterize with conventional task-level prompt evaluation. This study proposes an observer-side diagnostic vocabulary for describing prompt-induced interference in large language models (LLMs). The vocabulary organizes prompt effects into four macro-groups—framing (role/task/audience/objective), reasoning (process/scope), expression (style/format/length), and epistemic control (stance/constraints)—and is instantiated here as the Z-model, an auditable 11-axis reference basis for reporting and comparison under black-box access. This reference basis is a pragmatic descriptive choice for interpretability and reporting, rather than an ontological, minimality, or model-internal claim. Empirically, we do not attempt to validate all four macro-groups at once. Instead, we run a targeted Japanese/English stress test of one high-leverage pathway: an expression-oriented politeness cue and its secondary effects on epistemic- and scope-related proxies. Under a matched interaction protocol (five benign topics; 250 samples per language-condition), the same politeness cue reliably changes expression while redistributing uncertainty and alternative/conditional markers in language-dependent ways. These are interpreted as protocol-level effects, and potential confounds from model training and alignment differences across languages are explicitly discussed. As a prediction-to-observation check, we additionally run a small factorial 2×2 probe and observe localized non-additivity consistent with structured latent interference. Key directional patterns are also reproduced on a pinned open-weight model checkpoint. Overall, the contribution is a scoped diagnostic framework plus evidence that one targeted cross-group pathway can be made auditable with lightweight black-box probes; the inverse direction is presented as a post-hoc diagnostic workflow rather than a validated latent estimator.
URL: https://openreview.net/forum?id=zcGznzAZwG
---
Title: Estimating and Auditing Binary LLM Decision Propensity under Restricted Closed APIs
Abstract: Many deployed LLM interfaces expose only a hard decision while hiding logits, hidden states, and internal confidence. We study how to audit binary decision propensity in this restricted setting. We model each prompt state by a latent binary log-odds margin and estimate that margin from repeated binary queries using a simple Beta-Bernoulli observer. A finite-budget receding-horizon controller is then used as a probe of prompt-side influence under partial observation. We validate the framework in two stages. In open-weight models, short prompt interventions induce systematic movement in a directly observed binary margin, and repeated binary samples recover that hidden state with useful fidelity. In the harder closed-API setting, admissible naturalistic prompting yields modest but nonzero movement, whereas policy injection provides a much stronger upper bound. On grounded StrategyQA, with-context naturalistic attack rises from 10.70% on GPT-4o-mini to 24.91% on GPT-5-nano; on grounded BoolQ, the corresponding rates are 6.49% and 21.67%. Removing context systematically increases attackability, and the full closed-loop method improves over fixed-prompt and one-step baselines, though at higher query cost. Overall, the results support a measurement-first claim: repeated binary observations are sufficient to audit an interface-level decision propensity under restricted APIs and to characterize how stable, marginal, or effort-intensive binary LLM decisions are in deployment-relevant settings.
URL: https://openreview.net/forum?id=TY54P5DfCb
---
Title: Fixed-Point Probing for GNN Depth Diagnostics: A Geometry-Consistent Protocol with a Patent-Citation Case Study
Abstract: Deep graph neural networks (GNNs) often degrade with depth, but endpoint metrics and any single probe do not reveal whether late-depth behavior reflects benign stabilization, classical oversmoothing, or a geometry-specific failure mode. Here, we read depth as a sequence of learned representations, not just as a model-size hyperparameter.
We introduce fixed-point probing, a post-training protocol that keeps the probe subset fixed and the measurements geometry-consistent, so familiar signals can be read together across depths and embedding geometries.
Applied to depth sweeps up to 32 layers on a patent-citation stress test, the protocol reveals geometry-dependent late-depth regimes. Euclidean models exhibit gradual class-structure degradation consistent with classical oversmoothing, while hyperbolic models enter a late-depth regime in which representation drift and graph-local roughness increase as embeddings approach the boundary. A tuned hyperbolic control matches Euclidean performance at shallow depth yet exhibits the same qualitative late-depth pattern, indicating that this effect is not explained by a trivially weak baseline.
Taken together, the results point to a boundary-coupled late-depth regime in hyperbolic GNNs that is hard to isolate from endpoint metrics or from any single probe alone, but becomes visible when the probes are read jointly under a shared protocol. The protocol is the main contribution; the patent citation graph is used as a stress-test case study, not as evidence for dataset-universal claims.
URL: https://openreview.net/forum?id=poycUyGplw
---
Title: Parallelizable Neural Turing Machines
Abstract: We introduce a parallelizable simplification of Neural Turing Machine (NTM), referred to as P-NTM, which reformulates the core operations of the original architecture as associative operators, enabling the use of efficient parallel scan algorithms. We additionally develop a log-space parallel algorithm for the numerically stable computation of these operations over long sequences. We evaluate the proposed architecture on a synthetic benchmark of algorithmic problems involving state tracking, memorization, and basic arithmetic, solved via autoregressive decoding. We compare P-NTM against a revisited stable implementation of the standard NTM, as well as conventional recurrent and attention-based architectures. Results show that, despite its simplifications, the proposed architecture matches the original in generalization on all evaluated tasks, solving all problems with perfect accuracy, including at unseen sequence lengths. We argue that this is achieved by replacing the recurrent controller with autoregressive control through output tokens. It also exhibits superior training efficiency, with parallel execution being up to an order of magnitude faster than the standard NTM. Ultimately, this work contributes toward the development of efficient neural architectures capable of expressing a broader class of algorithms.
URL: https://openreview.net/forum?id=CopkFS0um0
---
Title: MetaCog-Bench: A Process-Based Benchmark for Evaluating Metacognitive Monitoring and Control in Large Language Models
Abstract: We introduce MetaCog-Bench, a benchmark for evaluating metacognitive monitoring and control in large language models, grounded in the Nelson & Narens (1990) framework. Unlike prior benchmarks that rely on LLM-as-judge evaluation---which inflates scores when the same model family serves as both subject and evaluator---MetaCog-Bench uses exclusively deterministic evaluation: regex matching, keyword detection, JSON field verification, and Expected Calibration Error (ECE) computation. The benchmark comprises 147 tasks organized into five tiers spanning three metacognitive dimensions: Metacognitive Sensitivity (MS), Strategy Adaptation Frequency (SAF), and Cross-Domain Transfer Coefficient (CDTC). We evaluate seven models from six providers---including five proprietary frontier models, one proprietary mid-tier model, and one open-weight model (12B)---with three runs per model for statistical rigor. Grok-3-mini-fast achieves the highest overall score (0.864±0.009) with perfect metacognitive control (SAF=1.000), while DeepSeek-V3 follows closely (0.859±0.007) with the best confidence calibration (ECE=0.050). GPT-4o exhibits a striking monitoring-control dissociation: strong calibration (ECE=0.069) but weak sycophancy resistance (91.7%) and domain transfer (65.0%). The open-weight Open-Mistral-Nemo (12B) scores 0.710±0.026 overall but achieves near-proprietary sycophancy resistance (SAF=0.956), suggesting some metacognitive capabilities do not require frontier-scale models. All models achieve ≥96% ecological validity with unconstrained prompts versus ≤40% under JSON format constraints, demonstrating that structured output formats suppress metacognitive expression. A systematic keyword evaluation audit (100 sampled responses) validates the deterministic scoring pipeline at >96% accuracy.
URL: https://openreview.net/forum?id=ayRu5RL4Hb
---
Title: Provable Multi-Region Affinity Enforcement and Constraint Satisfaction for Scientific Machine Learning
Abstract: Neural networks have shown strong promise in scientific machine learning, but strictly enforcing boundary, interface, and safety constraints remains difficult. Soft penalties require careful tuning and do not guarantee exact satisfaction, while existing hard-constraint methods are typically specialized to particular equations or geometries. We introduce mPOLICE, a general framework that guarantees exact constraint satisfaction across multiple disjoint regions by exploiting the piecewise-linear structure of standard neural networks. By strategically configuring the network's internal activations, mPOLICE ensures that the learned function becomes exactly affine (linear) throughout each user-specified constrained zone. Once the network is locally affine, complex physical and safety constraints reduce to simple linear equations evaluated only at the corners (vertices) of each zone. Crucially, our approach handles many disjoint regions independently--a major limitation of existing single-region approaches. The method adds no inference-time overhead, requires no architectural changes, and integrates readily with standard training pipelines. We validate mPOLICE on operator learning, PDE boundary-condition enforcement, safety-critical control, and implicit 3D shape modeling.
URL: https://openreview.net/forum?id=2qXQcZxnAq
---
Title: Open Vocabulary Compositional Explanations for Neuron Alignment
Abstract: Compositional explanations leverage logical relationships between concepts to express the spatial alignment between neuron activations and human knowledge. However, these explanations rely on human-annotated datasets, restricting their applicability to specific domains and predefined concepts. This paper addresses this limitation by introducing a framework for the vision domain that allows users to probe neurons for arbitrary concepts and datasets. Specifically, the framework leverages open vocabulary semantic segmentation models to compute open vocabulary compositional explanations. The proposed framework consists of three steps: identifying concept sets, generating semantic segmentation masks using open vocabulary models, and deriving compositional explanations from these masks. The paper also proposes a process that leverages semantic knowledge graphs to analyze and compare compositional explanations computed by different methods sharing the same setup. The paper compares the proposed framework with previous methods for computing compositional explanations both in terms of quantitative metrics and human interpretability, analyzes the differences in explanations when shifting from human-annotated data to model-annotated data, and showcases the additional capabilities provided by the framework in terms of flexibility of the explanations with respect to the tasks and properties of interest.
URL: https://openreview.net/forum?id=iS38vzTMdd
---
Title: MoMamba: A Lightweight Music Oriented Mamba-Based Model for Music Information Retrieval Tasks
Abstract: Music Information Retrieval (MIR) tasks on raw audio have traditionally been tackled using convolutional neural networks (CNNs) and transformer-based models. While CNNs effectively capture local structures and transformers leverage attention for long-range dependencies, both architectures come with computational and scalability challenges. Recently, State-Space models, such as Mamba, have become popular for long time series data such as audio, and have shown considerable promise compared to convolutional or transformer-based architectures. In this study, we introduce a novel extension of Mamba tailored to music. Our resulting method, MoMamba (Music Oriented Mamba), is a lightweight Mamba-based music classification model. We evaluate MoMamba's performance across several benchmark MIR tasks. Our results show that MoMamba consistently outperforms a number of baselines, including an existing Mamba-based method, on all of the benchmark datasets we considered. Importantly, all models were trained from scratch without any pretraining, making the performance gains especially notable since they cannot be attributed to transfer learning. Additionally, our model's performance rivals existing benchmarks from models pretrained on much larger datasets. Our work highlights the advantages of MoMamba in music analysis and retrieval such as accuracy and inference time, encouraging further research into its capabilities within the MIR domain.
URL: https://openreview.net/forum?id=ywtLPNydZs
---