Weekly TMLR digest for Jul 12, 2026
TMLR
New certifications
==================
J2C Certification: CHyLL: Learning Continuous Neural Representations of Hybrid Systems
Sangli Teng, Hang Liu, Jingyu Song, Koushil Sreenath
https://openreview.net/forum?id=xK4WQnf7Yj
---
J2C Certification: VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wild
Junlin Fang, Wenyu Chen, Reshmi Ghosh, Robert Sim, Ahmed Salem, Vitor R. Carvalho, Emily Lawton, Sharon Li, Jack W. Stokes, Sean Du
https://openreview.net/forum?id=z7gczmhmmo
---
Survey Certification: Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Models
Yubo Li, Xiaobin Shen, Yidi Miao, Xinyu Yao, Xueying Ding, Ramayya Krishnan, Rema Padman
https://openreview.net/forum?id=UYNQXPevpF
---
Accepted papers
===============
Title: Sliding Window Recurrences for Sequence Models
Authors: Dragos Secrieru, Garyk Brixi, Yoshua Bengio, Taiji Suzuki, Michael Poli, Stefano Massaroli
Abstract: Multi-hybrid architectures are poised to take over language modeling due to better quality
and performance. We introduce a hierarchical decomposition framework for linear recur-
rences that yields Sliding Window Recurrences, a windowed training mode for recurrence
layers in hybrid models. Unlike sliding-window attention, SWR is derived from the transfer
structure of recurrences: it truncates the carrier system induced by the decomposition while
preserving dense local recurrence dynamics. We focus specifically on hardware-aligned win-
dows which are naturally jagged, limiting costly inter-warp communication. Using SWR,
we develop Phalanx layers for hybrid language models. In 1B parameter multi-hybrid mod-
els, Phalanx achieves over 10-40% speedup across 4K to 16K context length over optimized
Transformers while matching perplexity.
URL: https://openreview.net/forum?id=V09uO70ouz
---
Title: RA-CoA: Training-free Fashion Image Captioning via Retrieval-Augmented Chain-of-Attributes
Authors: Abhirama Subramanyam Penamakuri, Shreya Shukla, Anand Mishra
Abstract: Fashion Image Captioning (FIC) plays a vital role in enhancing user experience and product search in e-commerce platforms. Unlike natural scene image captioning, FIC requires fine-grained visual reasoning and knowledge of domain-specific terminology to capture subtle attributes such as neckline and closure types, graphic patterns, and dress silhouettes. Moreover, as fashion inventories evolve rapidly with new trends, styles, and frequently emerging vocabulary, developing training-free captioning solution becomes essential for scalability and real-world adaptability. Instruction-tuned vision-language models (VLMs) offer a promising solution to fashion image captioning dueto their strong zero-shot capabilities and natural language fluency. However, these general-purpose models often lack attribute-level coverage and precision, and tend to hallucinate or misidentify fine-grained fashion details, making them less suitable for high-fidelity applications like product cataloging or personalized recommendations. To address this, we propose RA-CoA (Retrieval-Augmented Chain-of-Attributes), a novel, training-free framework that disentangles fashion image captioning into two interpretable stages: (i) retrieval of relevant attribute sets from a product knowledge base, and (ii) attribute-level reasoning to generate the final caption. RA-CoA is a model-agnostic approach that works with frozen VLMs to improve fine-grained attribute precision in product captions without the need for fine-tuning. Extensive evaluations across diverse VLM model families under different prompting paradigms demonstrate that RA-CoA significantly improves caption quality, achieving an average gain of 26.3% METEOR score over zero-shot captioning. We make our code publicly available.
URL: https://openreview.net/forum?id=PpkOrVUpJ6
---
Title: A Monte Carlo Framework for Calibrated Uncertainty Estimation in Sequence Prediction
Authors: Qidong Yang, Weicheng Zhu, Joseph Keslin, Laure Zanna, Tim G. J. Rudner, Carlos Fernandez-Granda
Abstract: Probabilistic prediction of sequences from images and other high-dimensional data remains a key challenge---particularly in safety-critical domains. In these settings, it is often desirable to quantify the uncertainty associated with a prediction in addition to determining the most likely sequence. In this paper, we consider a Monte Carlo framework to estimate probabilities and confidence intervals associated with sequences. The framework uses a Monte Carlo simulator, implemented as an autoregressively trained neural network, to sample sequences conditioned on an image input. We then use these samples to estimate probabilities and confidence intervals, which are then evaluated using a suite of customized uncertainty evaluation metrics. Experiments on synthetic and real data show that the framework produces accurate discriminative predictions, but can suffer from miscalibration. To address this shortcoming, we propose a time-dependent regularization method, which produces calibrated predictions.
URL: https://openreview.net/forum?id=sJE59flFC1
---
Title: When Does Multimodality Lead to Better Time Series Forecasting?
Authors: Xiyuan Zhang, Boran Han, Haoyang Fang, Abdul Fatir Ansari, Shuai Zhang, Danielle C. Maddix, Cuixiong Hu, Andrew Gordon Wilson, Michael W. Mahoney, Hao Wang, Yan Liu, Michael Bohlke-Schneider, Huzefa Rangwala, George Karypis, Bernie Wang
Abstract: Recently, there has been growing interest in incorporating textual information into foundation models for time series forecasting. However, it remains unclear whether and under what conditions such multimodal text integration consistently yields gains. We systematically investigate these questions across a diverse benchmark of 16 forecasting tasks spanning 7 domains, including health, environment, and economics. We evaluate two popular multimodal forecasting paradigms: aligning-based methods, which align time series and text representations; and prompting-based methods, which directly prompt large language models for forecasting. Our findings reveal that the benefits of text integration are highly condition-dependent. While we confirm reported gains in some settings, these improvements are not universal across datasets or models. To move beyond empirical observations, we disentangle the effects of model architectural properties and data characteristics, drawing data-agnostic insights that generalize across domains. Our findings highlight that on the modeling side, incorporating text information is most helpful given (1) high-capacity text models, (2) comparatively weaker time series models, and (3) appropriate aligning strategies. On the data side, performance gains are more likely when (4) sufficient training data is available and (5) the text offers complementary predictive signal beyond what is already captured from the time series alone. Our study offers a rigorous, quantitative foundation for understanding when textual context can be expected to aid forecasting tasks, and reveals that its benefits are neither universal nor always aligned with intuition.
URL: https://openreview.net/forum?id=RggcWYWR3N
---
Title: Low-Rank Filtering & Smoothing for Sequential Deep Learning
Authors: Joanna Sliwa, Frank Schneider, Nathanael Bosch, Agustinus Kristiadi, Philipp Hennig
Abstract: Learning multiple tasks sequentially requires neural networks to balance retaining knowledge, yet being flexible enough to adapt to new tasks. Regularizing network parameters is a common approach, but it rarely incorporates prior knowledge about task relationships, and limits information flow to future tasks only. We propose a Bayesian framework that treats the network's parameters as the state space of a nonlinear Gaussian model, unlocking two key capabilities: (1) A principled way to encode domain knowledge about task relationships, allowing, e.g., control over which layers should adapt between tasks. (2) A novel application of Bayesian smoothing, allowing task-specific models to also incorporate knowledge from models learned later. This does not require direct access to their data, which is crucial, e.g., for privacy-critical applications. These capabilities rely on efficient filtering and smoothing operations, for which we propose diagonal plus low-rank approximations of the precision matrix in the Laplace approximation (LR-LGF). Empirical results demonstrate the efficiency of LR-LGF and the benefits of the unlocked capabilities.
URL: https://openreview.net/forum?id=1TJXpLHLKG
---
Title: Hierarchy-Aware Multimodal Unlearning for Medical AI
Authors: Fengli Wu, Vaidehi Patil, Jaehong Yoon, Yue Zhang, Mohit Bansal
Abstract: Multimodal large language models (MLLMs) are increasingly used in sensitive domains such as medical AI, where privacy regulations, including HIPAA and GDPR, require the removal of specific individuals' or institutions' data. This motivates machine unlearning, which aims to remove the influence of target data from a trained model. However, existing unlearning benchmarks fail to reflect the hierarchical and multimodal structure of real-world medical data, limiting their ability to properly evaluate unlearning in practice. Therefore, we introduce MedForget, a hierarchy-aware multimodal unlearning benchmark that models data from medical institutions as a nested structure, enabling fine-grained evaluation across retain and forget splits. Experiments show that current unlearning methods struggle to achieve effective hierarchy-aware forgetting without degrading retained utility, measured by performance on clinically relevant prediction tasks. To address this limitation, we propose Cross-modal Hierarchy-Informed Projection for unlearning (CHIP), a training-free, hierarchy-aware multimodal unlearning method that deletes information by selectively removing target-specific weight subspaces while preserving sibling-shared information. Our results show that CHIP achieves the highest forget-retain performance gap across all hierarchy levels while maintaining competitive downstream utility compared to existing methods. Overall, MedForget provides a practical benchmark for evaluating hierarchy-aware multimodal unlearning for medical data, while CHIP offers an effective solution that generalizes across architectures and balances deletion with utility.
URL: https://openreview.net/forum?id=TVSIhLqIkf
---
Title: Data-Dependent Regret and Polyak Corrections for Constrained Online Convex Optimization
Authors: Wentao Zhang
Abstract: In constrained online convex optimization, the learner must minimize regret against adversarially chosen convex costs while satisfying a convex constraint at every round, a requirement that arises naturally in safety-critical domains such as power systems, autonomous control, and clinical decision-making. A natural and computationally efficient approach augments online gradient descent with a Polyak feasibility step: a closed-form half-space projection requiring only one constraint evaluation and one subgradient per round. This approach is known to achieve $O(\sqrt{T})$ regret with per-round feasibility, yet we prove that its existing analysis is strictly loose by identifying two quantities it unnecessarily discards. Specifically, replacing the worst-case gradient envelope $G_f^2 T$ with the observed accumulation $\mathcal{G}_T = \sum_t \lVert \nabla f_t(x_t) \rVert^2$ yields a data-dependent bound without any algorithmic modification. Furthermore, we introduce the Polyak correction $\mathcal{P}_T \geq 0$, which captures the cumulative squared displacement of the feasibility projection and enters the regret bound with a strictly negative sign, a term that all prior proofs lose entirely through the Pythagorean inequality. The total improvement $\Delta_T = \tfrac{\eta}{2}(G_f^2 T - \mathcal{G}_T) + \tfrac{1}{2\eta}\mathcal{P}_T$ is provably non-negative and decomposes into two independent, complementary sources that vanish only in a degenerate corner case. Building on these analytical insights, we propose AdaOGD-PFS, an adaptive-step-size variant that achieves $O(\sqrt{\mathcal{G}_T})$ regret, potentially much smaller than $O(G_f\sqrt{T})$, while preserving per-round constraint satisfaction. Experiments on ball-constrained and halfspace-constrained instances confirm bound improvements of 38--43%, with both data-dependent gradients and Polyak corrections contributing meaningfully.
URL: https://openreview.net/forum?id=tBM4UrAPf8
---
Title: LENS: Learning to Navigate with Active Search for Partially Observable MAPF in Unknown Environments
Authors: Di Yang, Yuheng Li, Yanhai Xiong
Abstract: Classical Multi-Agent Path Finding (MAPF) solvers guarantee collision-free coordination but rely on perfect global knowledge, limiting their applicability in strictly unknown environments. Consequently, modern learning-based approaches face a dichotomy: decentralized reactive heuristics scale under partial observability but fail at structured deadlocks due to limited horizons and weak interaction inductive biases, while neural foundation models (e.g., MAPF-GPT) provide topological awareness but require pre-computed global heuristics and prohibitive training data.
We address Centralized Collaborative Partially Observable MAPF (PO-MAPF) by proposing LENS (\textbf{LE}arning to \textbf{N}avigate with active \textbf{S}earch), a hybrid architecture decoupling topological guidance from local collision avoidance. LENS employs a lightweight neural network that maps each agent's field of view (FOV) and goal direction into a dense local potential field to generate multi-step subpaths. Upon anticipating collisions, agents aggregate local observations into a shared belief, projecting neural proposal endpoints as waypoints. LENS then partitions anticipated conflicts into disjoint graphs and invokes localized Conflict-Based Search (L-CBS) over bounded spatio-temporal windows. This ensures strictly collision-free execution within a receding horizon, overcoming reactive myopia without the exponential overhead of global replanning.
Evaluations show that given full global knowledge, LENS approximates the solution quality of centralized oracles across most benchmarks, and achieves out-of-distribution generalization comparable to 85M-parameter foundation models using $<0.2\%$ of their training data. In strictly unknown environments with online mapping, LENS outperforms reactive baselines, improving success rates by $41.5\%$ in high-density mazes. By decoupling topological inference from collision resolution, LENS provides a scalable, data-efficient, and locally collision-free solution for autonomous navigation.
URL: https://openreview.net/forum?id=DuNU6ZN5GG
---
Title: Hallucination Detection and Mitigation with Diffusion in Multi-Variate Time-Series Foundation Models
Authors: Vijja Wichitwechkarn, Charles Fox, R. Choudhary
Abstract: Foundation models (FMs) for natural language processing have many coherent definitions of hallucination and methods for its detection and mitigation. However, analogous definitions and methods do not exist for multi-variate time-series (MVTS) FMs. We propose new definitions for MVTS hallucination, along with new detection and mitigation methods using a diffusion model to estimate hallucination levels. We derive relational datasets from popular time-series datasets to benchmark these relational hallucination levels. Using these definitions and models, we find that open-source pre-trained MVTS imputation FMs relationally hallucinate on these datasets on average up to 59.5\% as much as a weak baseline. The proposed mitigation method reduces this by up to 47.7\% for these models. The definition and methods may improve adoption and safe usage of MVTS FMs.
URL: https://openreview.net/forum?id=fHGQ7hZlb5
---
Title: Conditional Risk-Averse Constrained Reinforcement Learning
Authors: James McCarthy, Radu Marinescu, Elizabeth M. Daly, Ivana Dusparic
Abstract: In Risk-averse Constrained Reinforcement Learning (RaCRL), the optimal tolerance for risk often depends on a preference over the trade-off between reward and safety. This trade-off is influenced by environmental uncertainty, which is generally difficult to quantify, in turn making its effect on an agent's performance difficult to predict at the outset of training. Conventional RaCRL approaches typically train agents under a fixed risk level, set at the beginning of training, leading to an agent with a fixed, often conservative, reward-safety trade-off at deployment time. In this paper, we introduce Conditional Risk-averse Actor Critic (CRAC), a novel algorithm for RaCRL that conditions the agent on risk levels sampled during both exploration and learning. Through exploring and learning from diverse experiences across varied risk levels, CRAC generalises effectively across a spectrum of risk preferences, enabling the deployment of a single agent at risk levels chosen by a user. We evaluate CRAC across a set of environments with increasing difficulty, demonstrating empirically that it generalises effectively across a risk spectrum. CRAC often achieves higher reward than fixed-risk agents, whilst satisfying cost constraints. In cases where CRAC's reward performance is marginally lower than a fixed-risk agent, CRAC retains the advantage of a single risk-conditioned policy that generalises to a risk spectrum, reducing training overhead and providing more control over the reward-cost trade-off.
URL: https://openreview.net/forum?id=JJFFx1HVHi
---
Title: Exposing Long-Tail Safety Failures in Large Language Models through Efficient Diverse Response Sampling
Authors: Suvadeep Hajra, Palash Nandi, Tanmoy Chakraborty
Abstract: Safety tuning through supervised fine-tuning and reinforcement learning from human feedback has substantially improved the robustness of large language models (LLMs). However, it typically suppresses rather than eliminates unsafe behaviors, leaving rare but critical failures hidden in the long tail of the output distribution. While most red-teaming work emphasizes adversarial prompt search (input-space search), we show that these hidden risks can be systematically exposed through diverse response generation (output-space search). Specifically, we show that, for a fixed safety-critical prompt, increasing the number and diversity of sampled responses monotonically raises the jailbreak success rate. To efficiently uncover these failures, we propose Progressive Diverse Population Sampling (PDPS). This approach replaces naive, large-scale IID sampling with a multi-stage expansion-and-selection strategy that generates a compact, semantically diverse set of responses at a substantially lower computational cost. Across multiple jailbreak benchmarks and open-source LLMs, PDPS achieves attack success rates comparable to large-scale IID sampling while using only 8%-29% of the computational cost, and outperforms IID sampling and Diverse Beam Search by 26%-40% under limited-response budgets, while uncovering a broader and more semantically diverse range of failure modes. Critically, this diversity translates directly into more effective safety hardening: when integrated into an RLHF-based safety-tuning pipeline, PDPS-generated unsafe responses yield 33% and 41% greater reductions in ASR than those generated by IID sampling and Diverse Beam Search, respectively. Finally, we show that while input-space prompt optimization methods fall short of output-space exploration when used in isolation, combining input-space perturbation with diversity-driven output-space exploration covers a wider range of failure modes more efficiently than either paradigm alone.
URL: https://openreview.net/forum?id=tHfAskovWI
---
Title: Aggregation-Free Heterogeneous Federated Learning with Data-Free Knowledge Exchange
Authors: Haowen Guan, Xuan Zhao, Chengjie Zheng, Allen Yang, Tales Imbiriba, Shichao Pei
Abstract: Heterogeneous Federated Learning (HFL) is a decentralized machine learning paradigm that enables participants to leverage distributed knowledge from diversified environments while safeguarding individual privacy. Recent works that address both data and model heterogeneity still require aggregating model parameters, which restricts architectural flexibility. Knowledge Distillation (KD) has been adopted in HFL to circumvent direct model aggregation by aggregating knowledge, but it depends on a public dataset and may incur information loss when redistributing knowledge from the global model. We propose Federated Knowledge Exchange (FKE), an aggregation-free FL paradigm in which each client acts as both teacher and student, exchanging knowledge directly with peers and removing the need for a global model. To remove reliance on public data, we attach a lightweight embedding decoder that produces transfer data, forming the Data-Free Federated Knowledge Exchange (DFFKE) framework. Extensive experiments show that DFFKE surpasses nine state-of-the-art HFL baselines by up to 18.14%. Code: https://github.com/HaowenGuan/DFFKE.
URL: https://openreview.net/forum?id=BLQ9JulhQm
---
Title: Regularity and Stability Properties of Selective SSMs with Discontinuous Gating
Authors: Nikola Zubic, Davide Scaramuzza
Abstract: Selective State-Space Models (SSMs) such as Mamba have become central to long-sequence modeling. Still, their stability is poorly understood: their state-space coefficients are modulated online by a token-dependent gating signal, making the recurrence neither linear time-invariant nor classically nonlinear. We study continuous-time selective SSMs through passivity, dissipativity, and Input-to-State Stability (ISS), explicitly separating the selection signal $x(\cdot)$ from the driving input $u(\cdot)$. We obtain four results: exponential forgetting under strict dissipativity; a canonical $\mathrm{AUC}_{\mathrm{loc}}$ quadratic storage for the frozen-selection subsystem that accommodates discontinuous gating; a parametric LMI together with universal kernel constraints and "irreversible forgetting" under universal quadratic storage; and sufficient conditions for global ISS uniformly over admissible selection schedules. We then bridge to practice by deriving a sampled block LMI for the Mamba selective-scan core, which is used as a differentiable training-time regularizer. Across seven standard time-series datasets and four prediction horizons, the regularizer reduces sampled Mamba-core LMI violations by roughly 92% in 28/28 pairs at a clean-MSE cost of less than 0.018%. It improves internal Mamba passivity and state-norm diagnostics under injected perturbations. Our results turn classical control-theoretic tools into verifiable structural and training criteria for selective SSMs, while honestly scoping which guarantees transfer to a deep selective-scan architecture.
URL: https://openreview.net/forum?id=7Vav53cDeN
---
Title: Progressive Checkerboards for Autoregressive Multiscale Image Generation
Authors: David Eigen
Abstract: A key challenge in autoregressive image generation is to efficiently sample independent locations in parallel, while still modeling mutual dependencies with serial conditioning. Some recent works have addressed this by conditioning between scales in a multiscale pyramid. Others have looked at parallelizing samples in a single image using regular partitions or randomized orders. In this work we examine a flexible, fixed ordering based on progressive checkerboards for multiscale autoregressive image generation. Our ordering draws samples in parallel from evenly spaced regions at each scale, maintaining full balance in all levels of a quadtree subdivision at each step. This enables effective conditioning both between and within scales. Intriguingly, we find evidence that in our balanced setting, a wide range of scale-up factors lead to similar results, so long as the total number of serial steps is constant. On class-conditional ImageNet, our method achieves competitive performance compared to recent state-of-the-art autoregressive systems with like model capacity, using fewer sampling steps.
URL: https://openreview.net/forum?id=wkCTCXo1a9
---
Title: Lens: A Knowledge-Guided Foundation Model for Network Traffic
Authors: Xiaochang Li, Chen Qian, Qineng Wang, Jiangtao Kong, Yuchen Wang, Ziyu Yao, Bo Ji, Long Cheng, Gang Zhou, Huajie Shao
Abstract: Network traffic refers to the amount of data being sent and received over the Internet or any system that connects computers. Analyzing network traffic is vital for security and management, yet remains challenging due to the heterogeneity of plain-text packet headers and encrypted payloads. To capture the latent semantics of traffic, recent studies have adopted Transformer-based pretraining techniques to learn network representations from massive traffic data. However, these methods pre-train on data-driven tasks but overlook network knowledge, such as masking partial digits of the indivisible network port numbers for prediction, thereby limiting semantic understanding. In addition, they struggle to extend classification to new classes during fine-tuning due to the distribution shift. Motivated by these limitations, we propose Lens, a unified knowledge-guided foundation model for both network traffic classification and generation. In pretraining, we propose a Knowledge-Guided Mask Span Prediction method with textual context for learning knowledge-enriched representations. For extending to new classes in finetuning, we reframe the traffic classification as a closed-ended generation task and introduce context-aware finetuning to adapt the distribution shift. Evaluation results across various benchmark datasets demonstrate that the proposed Lens achieves superior performance on both classification and generation tasks. For traffic classification, Lens outperforms competitive baselines substantially on 8 out of 12 tasks with an average accuracy of 96.33% and extends to novel classes with significantly better performance. For traffic generation, Lens generates better high-fidelity network traffic for network simulation, gaining up to 30.46% and 33.3% better accuracy and F1 in fuzzing tests. Code is available at \url{https://github.com/V-Enzo/Lens-foundation-model-for-network-traffic}
URL: https://openreview.net/forum?id=cGDwTgnJIR
---
Title: Interpretable factorization of clinical questionnaires to identify latent factors of psychopathology
Authors: Ka Chun Lam, Francisco Pereira, Armin Raznahan, Bridget Wilson Mahony
Abstract: Psychiatry research seeks to understand the manifestations of psychopathology in behavior, as measured in questionnaire data, by identifying a small number of latent factors that explain them. While factor analysis is the canonical tool for this purpose, the resulting factors may not be interpretable, and may also be subject to confounding variables. Moreover, missing data are common, and explicit imputation is often required. To overcome these limitations, we introduce Interpretability Constrained Questionnaire Factorization (ICQF), a non-negative matrix factorization method with regularization tailored for questionnaire data. Our method aims to promote factor interpretability and solution stability. We provide an optimization procedure with theoretical convergence guarantees, and an automated procedure to determine latent dimensionality accurately. We validate these procedures using realistic synthetic data. We demonstrate the effectiveness of our method in a widely used general-purpose questionnaire, in two independent datasets (the Healthy Brain Network and Adolescent Brain Cognitive Development studies). Specifically, we show that ICQF preserves diagnostic information across a range of disorders, outperforming competing methods for smaller dataset sizes, and improves interpretability, as assessed by our clinical research collaborators and co-authors. This suggests that the regularization in our method matches domain characteristics, in addition to satisfying qualitative desiderata.
URL: https://openreview.net/forum?id=1Yq6INJwiO
---
Title: Learning to Strategically Acquire Resources in Competition
Authors: Safwan Hossain, Mirah Shi, Andrew Bennett, Neil Andrew Chriss, Michael Kearns, Anderson Schneider, Yuriy Nevmyvaka
Abstract: We consider multiple agents competing to acquire some costly divisible resource (\emph{e.g.} shares of a financial asset, compute resources, etc.) over time. Leveraging a standard model for price dynamics, we propose a novel game-theoretic model for this problem, generalizing settings studied in diverse literatures. Our analysis considers different assumptions on the information available to agents. Under partial-information with a common prior (which subsumes complete information as a special case), we establish the existence, uniqueness, and efficient computability of the Bayesian Nash equilibrium (BNE), and bound the price of anarchy. Next and more generally, we consider agents with no common prior learning to act optimally given realistic market feedback from repeated interactions. We provide sufficient conditions on agents doing simultaneous learning dynamics for last-iterate convergence to the BNE. For all settings, we provide detailed simulations based on real financial data to illustrate our theory and offer new insights on strategic behavior in the context of trading and resource acquisition.
URL: https://openreview.net/forum?id=Z8j6jGVxAZ
---
Title: Autoregressive Models for Small-Scale Knowledge Graph Generation
Authors: Thiviyan Thanapalasingam, Antonis Vozikis, Peter Bloem, Paul Groth
Abstract: Knowledge Graph (KG) generation requires models to learn complex semantic dependencies between triples while maintaining domain validity constraints. Unlike link prediction, which scores triples independently, generative models must capture interdependencies across entire subgraphs to produce semantically coherent structures. We present ARK (Auto-Regressive Knowledge Graph Generation), a family of autoregressive models that generate KGs by treating graphs as sequences of (head, relation, tail) triples. ARK learns implicit semantic constraints directly from data, including type consistency, temporal validity, and relational patterns, without explicit rule supervision. On the IntelliGraphs benchmark, our models achieve 89.2% to 100.0% semantic validity across diverse datasets while generating novel graphs not seen during training. We also introduce SAIL, a variational extension of ARK that enables controlled generation through learned latent representations, supporting both unconditional sampling and conditional completion from partial graphs. Our analysis reveals that model capacity (hidden dimensionality >= 64) is more critical than architectural depth for KG generation, with recurrent architectures achieving comparable validity to transformer-based alternatives while offering substantial computational efficiency. These results demonstrate that autoregressive models provide an effective framework for KG generation, with practical applications in knowledge base completion and query answering.
URL: https://openreview.net/forum?id=xhy0tB4uzb
---
Title: Reliability Scaling Laws for Quantized Large Language Models
Authors: Sirine Ayadi, Sándor Daróczi, Stephan Günnemann, Bertrand Charpentier
Abstract: Quantization is a powerful strategy to build capable and resource-efficient large language models (LLMs) by reducing the bitwidth of the parameters. While quantized LLMs achieve state-of-the-art performance on unperturbed inputs using standard predictive metrics, their performance on perturbed inputs, measured using reliability metrics, remains underexplored, despite its importance for reliable deployment. To address this gap, we conduct a comprehensive reliability evaluation of quantized LLMs consisting of three key components: (1) Uncertainty: We assess the trustworthiness of LLMs quantized to 2, 3, 4, and 8 bits using six different quantization methods, employing established uncertainty metrics. (2) Robustness: We design character-level and word-level input perturbations to evaluate the reliability of quantized models under semantically-preserving variations in the inputs that arise in real-world applications.
(3) Reliability scaling trends: We investigate how the reliability scales with the number of model bits. Our study reveals that while the performance scales monotonically with the total number of bits, the reliability scalings are nonlinear. A reliability peak occurs for 4-bit quantized models, indicating that quantizing moderately sized models offers the best reliability-efficiency trade-off. Additionally, our empirical findings reveal that quantization enhances the robustness of LLMs to natural input perturbations.
URL: https://openreview.net/forum?id=UUBijehMQO
---
Title: Mutual Information Collapse Explains Disentanglement Failure in $\beta$-VAEs
Authors: Minh Vu, Xiaoliang Wan, Shuangqing Wei
Abstract: The $\beta$-VAE is a foundational framework for unsupervised disentanglement, utilizing the regularization parameter $\beta$ to balance latent factorization against reconstruction fidelity. However, disentanglement performance often exhibits a non-monotonic dependence on $\beta$: standard metrics, such as MIG and SAP, typically peak at intermediate values and deteriorate under stronger regularization. We characterize this phenomenon as informational collapse—an information-theoretic failure in which excessive regularization drives the mutual information between latent variables and ground-truth generative factors toward zero. By analyzing the stationarity conditions in a linear-Gaussian setting, we prove that, for $\beta>1$, alternating optimization induces a spectral contraction of the encoder gain, causing its spectral norm to decay exponentially and latent–factor mutual information to vanish. Motivated by this failure mode, we investigate the $\lambda\beta$-VAE, which augments the objective with an auxiliary $L_2$ reconstruction penalty. Our analysis shows that this term modifies the encoder stationarity conditions and weakens the derived upper bound on encoder-gain contraction relative to the standard $\beta$-VAE. Experiments on dSprites, Shapes3D, and MPI3D-real indicate that configurations with $\lambda>0$ generally improve reconstruction accuracy and exhibit an attenuated decline in disentanglement performance relative to the $\lambda=0$ baseline, particularly under strong regularization.
URL: https://openreview.net/forum?id=P0E4LsRvGY
---
Title: Heterogeneous Matrix Factorization: When Features Differ by Datasets
Authors: Naichen Shi, Salar Fattahi, Raed Al Kontar
Abstract: In myriad statistical applications, data are collected from related but heterogeneous sources. These sources share some commonalities while containing idiosyncratic characteristics. One of the most fundamental challenges in such scenarios is to recover the shared and source-specific factors at scale. Despite the existence of a few heuristic approaches, a scalable algorithm with theoretical guarantees has yet to be established.
In this paper, we tackle the problem by proposing a method called Heterogeneous Matrix Factorization to separate the shared and unique factors for a class of problems. HMF maintains the orthogonality between the shared and unique factors by leveraging an invariance property in the objective. The algorithm is easy to implement and intrinsically distributed. On the theoretic side, we show that for the square error loss, HMF will converge into the optimal solutions, which are close to the ground truth.
HMF can be integrated with auto-encoders to learn nonlinear feature mappings. Through a variety of case studies, we show how HMF can be applied to video segmentation, time-series feature extraction, and recommender systems.
URL: https://openreview.net/forum?id=1BUB0I3Obx
---
Title: Koopman-informed recurrent neural networks
Authors: Erik Lien Bolager, Ana Čukarska, Iryna Burak, Zahra Monfared, Felix Dietrich
Abstract: Recurrent neural networks are a successful neural architecture for many time-dependent problems, including time series analysis, forecasting, and modeling of dynamical systems. In the context of dynamical systems, training with backpropagation through time can lead to challenges arising from exploding or vanishing gradients. In this contribution, we introduce Koopman-informed recurrent neural networks, a computational approach to construct all weights and biases of a recurrent neural network without using gradient-based methods. The approach is based on a combination of random feature networks and Koopman operator theory for dynamical systems. The hidden parameters of a single recurrent block are sampled at random, while the outer weights are constructed using extended dynamic mode decomposition. This approach alleviates some problems with backpropagation commonly related to recurrent networks. The connection to Koopman operator theory also allows us to start using results in this area to analyze recurrent neural networks. In computational experiments on time series, forecasting for chaotic dynamical systems, control problems, and on real-world data, we observe that with comparable forecasting accuracy, the training time of the Koopman-informed recurrent neural networks is significantly improved when compared to models trained with commonly used gradient-based methods.
URL: https://openreview.net/forum?id=KHsnxKYG6k
---
Title: CHyLL: Learning Continuous Neural Representations of Hybrid Systems
Authors: Sangli Teng, Hang Liu, Jingyu Song, Koushil Sreenath
Abstract: Learning the flows of hybrid systems with both continuous and discrete dynamics is challenging. The existing method learns the dynamics in each discrete mode, which suffers from the combination of mode switching and discontinuities in the flows. In this work, we propose CHyLL (Continuous Hybrid System Learning in Latent Space), which learns a continuous neural representation of a hybrid system without trajectory segmentation, event functions, or mode switching. The key insight of CHyLL is that the reset map glues the state space at the guard surface, reformulating the state space as a piecewise smooth quotient manifold where the flow becomes spatially continuous. Building upon these insights and the embedding theorems grounded in differential topology, CHyLL concurrently learns a singularity-free neural embedding in a higher-dimensional space and the continuous flow in it. We demonstrate that CHyLL can accurately predict the flow of hybrid systems with superior accuracy and identify their topological invariants. Finally, we apply CHyLL to the stochasticoptimal control problem.
URL: https://openreview.net/forum?id=xK4WQnf7Yj
---
Title: VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wild
Authors: Junlin Fang, Wenyu Chen, Reshmi Ghosh, Robert Sim, Ahmed Salem, Vitor R. Carvalho, Emily Lawton, Sharon Li, Jack W. Stokes, Sean Du
Abstract: Vision-language Models (VLMs) are essential for contextual understanding of both visual and textual information. However, their vulnerability to adversarially manipulated inputs presents significant risks, leading to compromised outputs and raising concerns about the reliability in VLM-integrated applications. Detecting these malicious prompts is thus crucial for maintaining trust in VLM generations. A major challenge in developing a safeguarding prompt classifier is the lack of a large amount of labeled benign and malicious data. To address the issue, we introduce VLMGuard, a novel learning framework that leverages the unlabeled user prompts in the wild for malicious prompt detection. These unlabeled prompts, which naturally arise when VLMs are deployed in the open world, consist of both benign and malicious information. To harness the unlabeled data, we present an automated maliciousness estimation score for distinguishing between benign and malicious samples within this unlabeled mixture, thereby enabling the training of a binary prompt classifier on top. Notably, our framework does not require extra human annotations and is robust to realistic prompt variations, offering strong flexibility and practicality for real-world applications. Extensive experiments show that VLMGuard achieves superior detection results, improving AUROC by 9.46% on average over the state-of-the-art method. Disclaimer: This paper may contain offensive examples; reader discretion is advised. Code is available at: https://github.com/radiolab-ntu/vlmguard.
URL: https://openreview.net/forum?id=z7gczmhmmo
---
Title: Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
Authors: Pei-Chi Pan, Yingbin Liang, Sen Lin
Abstract: Large Language Models (LLMs) demonstrate transformative potential, yet their reasoning remains inconsistent and unreliable. Reinforcement learning (RL)–based fine-tuning is a key mechanism for improvement, but its effectiveness is fundamentally governed by reward design. Despite its importance, the relationship between reward modeling and core LLM challenges—such as evaluation bias, hallucination, distribution shift, and efficient learning—remains poorly understood. This work argues that reward modeling is not merely an implementation detail but a central architect of reasoning alignment, shaping what models learn, how they generalize, and whether their outputs can be trusted. We introduce Reasoning-Aligned Reinforcement Learning (RARL), a reasoning-centric taxonomic perspective that organizes diverse reward paradigms for multi-step reasoning. Within this perspective, we present a taxonomy of reward mechanisms, analyze reward hacking as a pervasive failure mode, and examine how reward signals unify challenges ranging from inference-time scaling to hallucination mitigation. We further critically evaluate existing benchmarks, highlighting vulnerabilities such as data contamination and reward misalignment, and outline directions for more robust evaluation. By integrating fragmented research threads and clarifying the interplay between reward design and fundamental reasoning capabilities, this work provides a foundational roadmap for building reasoning models that are robust, verifiable, and trustworthy.
URL: https://openreview.net/forum?id=TDfrN1TbGH
---
Title: Adaptive Federated Learning via Dynamical System Model
Authors: Aayushya Agarwal, Lawrence Pileggi, Gauri Joshi
Abstract: Hyperparameter selection is critical for stable and efficient convergence of heterogeneous federated learning, where clients differ in computational capabilities, and data distributions are non-IID. Tuning hyperparameters is a manual and computationally expensive process as the hyperparameter space grows combinatorially with the number of clients. To address this, we introduce an end-to-end adaptive federated learning method in which both clients and central agents adaptively select their local learning rates and momentum parameters.
Our approach models federated learning as a dynamical system, allowing us to draw on principles from numerical simulation and physical design. Through this perspective, selecting momentum parameters equates to critically damping the system for fast, stable convergence, while learning rates for clients and central servers are adaptively selected to satisfy accuracy properties from numerical simulation. The result is an adaptive, momentum-based federated learning algorithm in which the learning rates for clients and servers are dynamically adjusted and controlled by a single, global hyperparameter. By designing a fully integrated solution for both adaptive client updates and central agent aggregation, our method is capable of handling key challenges of heterogeneous federated learning, including objective inconsistency and client drift. Importantly, our approach achieves fast convergence while being insensitive to the choice of the global hyperparameter, making it well-suited for rapid prototyping and scalable deployment. Compared to state-of-the-art adaptive methods, our framework is shown to deliver superior convergence for heterogeneous federated learning while eliminating the need for hyperparameter tuning both client and server updates.
URL: https://openreview.net/forum?id=ebDfXAklXg
---
Title: High precision PINNs in unbounded domains: application to singularity formulation in PDEs
Authors: Yixuan Wang, Ziming Liu, Zongyi Li, Anima Anandkumar, Thomas Y. Hou
Abstract: We investigate the high-precision training of Physics-Informed Neural Networks (PINNs) in unbounded domains, with a special focus on applications to singularity formulation in PDEs. We propose a modularized approach and study the choices of neural network ansatz, sampling strategy, and optimization algorithm. When combined with rigorous computer-assisted proofs and PDE analysis, the numerical solutions identified by PINNs, provided they are of high precision, can serve as a powerful tool for studying singularities in PDEs. For 1D Burgers equation, our framework can lead to a solution with very high precision, and for the 2D Boussinesq equation, which is directly related to the singularity formulation in 3D Euler and Navier-Stokes equations, we obtain a solution whose loss is 4 digits smaller than that obtained in \cite{wang2023asymptotic} with fewer training steps. We also discuss potential directions for pushing towards machine precision for higher-dimensional problems.
URL: https://openreview.net/forum?id=g34kZZAbT8
---
Title: Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Models
Authors: Yubo Li, Xiaobin Shen, Yidi Miao, Xinyu Yao, Xueying Ding, Ramayya Krishnan, Rema Padman
Abstract: Recent advances in large language models (LLMs) have substantially improved single-turn task performance, yet real-world applications increasingly demand sophisticated multi-turn interactions. This survey provides a structured review of recent progress in evaluating and enhancing multi-turn LLM interactions. Centered on a task-family-oriented taxonomy, spanning $\mathrm{(i)}$ instruction following in domains such as mathematics and coding, and $\mathrm{(ii)}$ conversational engagement in role-play, healthcare, education, and adversarial jailbreak settings, we systematically examine the challenges of maintaining context, coherence, fairness, and responsiveness across prolonged dialogues. We organize existing benchmarks and datasets into coherent categories reflecting the evolving landscape of multi-turn dialogue evaluation, and review a broad spectrum of enhancement methodologies, including model-centric strategies (in-context learning, supervised fine-tuning, reinforcement learning, and architectural innovations), external integration approaches (memory augmentation, retrieval-based methods, and knowledge graphs), and agent-based techniques for collaborative interaction. Finally, we identify open challenges and promising directions for future research to further improve the robustness and effectiveness of multi-turn LLM interactions. The companion repository is available at \url{https://github.com/yubol-bobo/Awesome-Multi-Turn-LLMs}.
URL: https://openreview.net/forum?id=UYNQXPevpF
---
Title: MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applications
Authors: Praveenkumar Kanithi, Clement Christophe, Marco AF Pimentel, Tathagata Raha, Prateek Munjal, Nada Saadi, Hamza A Javed, Svetlana Maslenkova, Nasir Hayat, Ronnie Rajan, Shadab Khan
Abstract: While Large Language Models (LLMs) achieve superhuman performance on standardized medical licensing exams, these static benchmarks have become saturated and increasingly disconnected from the functional requirements of clinical workflows. To bridge the gap between theoretical capability and verified utility, we introduce MEDIC, a comprehensive evaluation framework establishing leading indicators of clinical LLM competence across five dimensions. These upfront indicators reveal cross-benchmark capability gaps, such as the divergence between static knowledge retrieval and functional execution, that inform model selection before costly deployment-based evaluation. Beyond standard question-answering, we assess operational capabilities using deterministic execution protocols and a novel Cross-Examination Framework (CEF), which quantifies information fidelity and hallucination rates without reliance on reference texts. Our evaluation across a heterogeneous task suite exposes critical performance trade-offs: we identify a significant knowledge-execution gap, where proficiency in static retrieval does not predict success in operational tasks such as clinical calculation or SQL generation. Furthermore, we observe a divergence between passive safety (refusal) and active safety (error detection), revealing that models fine-tuned for high refusal rates often fail to reliably audit clinical documentation for factual accuracy. These findings demonstrate that no single architecture dominates across all dimensions, highlighting the necessity of a portfolio approach to clinical model deployment. We accompany this work with a publicly available MEDIC leaderboard at \url{https://hf.co/spaces/m42-health/MEDIC-Benchmark}.
URL: https://openreview.net/forum?id=pDQe9Icwb6
---
Title: Towards Scalable and Robust Filtration Learning for Point Clouds via Principal Persistence Measure
Authors: Kaifeng Zhang, Kai Ming Ting
Abstract: Topological features in persistent homology extracted via a filtration process have been shown to enhance the performance of machine learning tasks on point clouds. The performance is highly related to the choice of filtration, thereby underscoring the critical significance of filtration learning. However, the current supervised filtration learning method for point clouds cannot scale well. We identify that this shortcoming stems from the utilization of Persistence Diagrams (PD) for encoding topological features, such as connected components, rings or voids, etc. To address this issue, we propose to use Principal Persistence Measure (PPM), an existing statistical approximation of PD, as an alternative representation and adapt existing network for PPM-based filtration learning. Experimental results on point cloud classification tasks show that, in controlled filtration learning settings, the PPM-based framework preserves comparable classification performance in several cases while substantially improving the scalability for larger point clouds and degrading more gradually under outlier contamination.
URL: https://openreview.net/forum?id=8NkZxJFQ3I
---
Title: From Tables to Time: Extending TabPFN-v2 to Time Series Forecasting
Authors: Shi Bin Hoo, Samuel Müller, David Salinas, Frank Hutter
Abstract: Recent progress in foundation models has enabled strong zero-shot performance for time series forecasting. In this work, we show that such capabilities can also emerge from tabular foundation models. We introduce TabPFN-TS, a simple method that treats forecasting as a tabular regression problem by combining lightweight temporal featurization with the pretrained TabPFN-v2. This formulation requires no time-series–specific pretraining and naturally supports both univariate and covariate-informed forecasting. Despite its compact size (11M parameters), TabPFN-TS achieves state-of-the-art performance on covariate-informed forecasting and competitive accuracy on univariate forecasting across the GIFT-Eval and fev-bench benchmarks. We further provide controlled analyses examining how the model interprets temporal structure, how featurization choices affect accuracy, and how forecasts change under alternative tabular backbones. Together, our results demonstrate that tabular foundation models—when paired with suitable temporal features—offer an efficient and versatile alternative for forecasting, bridging tabular and time-series learning within a unified framework.
URL: https://openreview.net/forum?id=KIkQj8VOUY
---
Title: Favorability of Loss Landscape with Regularization Requires Both Large Overparametrization and Initialization
Authors: Etienne Boursier, Matthew Bowditch, Matthias Englert, Ranko Lazic
Abstract: The optimization of neural networks under weight decay remains poorly understood from a theoretical standpoint. While weight decay is standard practice in modern training procedures, most theoretical analyses focus on unregularized settings. In this work, we investigate the loss landscape of the $\ell_2$-regularized training loss for two-layer ReLU networks. We show that the landscape becomes favourable -- i.e., spurious local minima represent a negligible fraction of local minima -- under large overparametrization, specifically when the network width $m$ satisfies $m \gtrsim \min(n^d, 2^n)$, where $n$ is the number of data points and $d$ the input dimension. More precisely in this regime, almost all constant activation regions contain a global minimum and no spurious local minima. We further show that this level of overparametrization is not only sufficient but also necessary via the example of orthogonal data. Finally, we demonstrate that such loss landscape results primarily hold relevance in the large initialization regime. In contrast, for small initializations -- corresponding to the feature learning regime -- optimization can still converge to spurious local minima, despite the favourability of the landscape.
URL: https://openreview.net/forum?id=jbU0Tjjhfg
---
Title: LoraQuant: Mixed-Precision Quantization of LoRA to Ultra- Low Bits for LLM Customization
Authors: Amir Reza Mirzaei, Yuqiao Wen, Yanshuai Cao, Lili Mou
Abstract: Low-Rank Adaptation (LoRA) has become a popular technique for parameter-efficient fine-tuning of large language models (LLMs). In many real-world scenarios, multiple adapters are loaded simultaneously to enable LLM customization for personalized user experiences or to support a diverse range of tasks. Although each adapter is lightweight in isolation, their aggregate cost becomes substantial at scale. To address this, we propose LoraQuant, a mixed-precision post-training quantization method tailored to LoRA. Specifically, LoraQuant reparameterizes each adapter by singular value decomposition (SVD) to concentrate the most important information into specific rows and columns. This makes it possible to quantize the important components to higher precision, while quantizing the rest to lower bitwidth. We conduct comprehensive experiments with LLaMA 2-7B, LLaMA 2-13B, and Mistral 7B models on mathematical reasoning, coding, and summarization tasks. Results show that our LoraQuant uses significantly lower bits than other quantization methods, but achieves comparable or even higher performance.
URL: https://openreview.net/forum?id=71svCWi178
---
Title: The Fake Mirror Effect: Foreign Feedback Disrupts Self-Correction in Minimal Recurrent Networks
Authors: Sungmoon Ong
Abstract: When a recurrent network iteratively refines its predictions, is the performance gain driven primarily by generic temporal integration, or does it depend on the specific geometry of the model's own prior output? To resolve this, we dissect a minimal 35-neuron recurrent network with controlled feedback interventions. Recurrent-specific gain is isolated by two contrasts that do not depend on the convergence-sensitive variable-noise residual: static repeated input, where any deterministic stateless aggregator yields zero gain by construction, and a 120k-parameter MNIST extension where the recurrent loop retains a small but reliable residual. Under variable noise a loop−ensemble residual is additionally present at low-to-moderate noise but is convergence-dependent (reported descriptively), with the no-feedback within-network ensemble matching or nominally exceeding the loop at high noise.
Testing feedback-source dependence directly, we replace self-generated feedback with structurally valid output from an independently trained clone. This consistently degrades performance relative to self-feedback at all tested scales. At the minimal scale, clone feedback drops accuracy below the no-feedback baseline — a scale-bounded fake-mirror inversion (robust under static input; marginal under variable noise). This absolute-harm penalty is scale-dependent in both regimes: under variable noise it transitions to a net-positive but still suboptimal signal at the larger tested widths and on MNIST, while under static input it attenuates toward near-neutral at the larger widths; coordinate-shuffled feedback by contrast is harmful at every tested scale.
Progressively stronger static aligners recover the clone-feedback loss essentially fully under static input but plateau near ~85% under variable noise, indicating that static open-loop mapping does not fully restore closed-loop compatibility across the aligner capacities we evaluated. A relative condition-number analysis does not separate the evaluated feedback-source conditions. Together, these results characterize feedback-geometry compatibility: an empirical relation between a recurrent receiver and the geometry of its self-generated feedback that is not captured by generic temporal integration alone.
URL: https://openreview.net/forum?id=cwENvGCLRv
---
Title: Recursive Entropic Risk Optimization in Discounted MDPs: Sample Complexity Bounds with a Generative Model
Authors: Oliver Mortensen, M. Sadegh Talebi
Abstract: We study risk-sensitive reinforcement learning in finite discounted infinite-horizon Markov Decision Processes with recursive entropic risk measures (ERM), where the risk parameter $\beta \neq 0$ controls the agent's risk attitude: $\beta>0$ for risk-averse behavior and $\beta<0$ for risk-seeking behavior. A generative model of the MDP is assumed to be available. Our focus is on the sample complexities of learning the optimal state–action value function (value learning) and a near-optimal policy (policy learning) under recursive ERM.
We introduce a model-based algorithm, called Model-Based ERM Q-Value Iteration (MB-ERM-QVI), and derive PAC-type bounds on its sample complexity for both value and policy learning. Both PAC bounds scale exponentially with $|\beta|/(1-\gamma)$, where $\gamma$ is the discount factor. We also establish corresponding lower bounds for both value and policy learning, showing that exponential dependence on $|\beta|/(1-\gamma)$ is unavoidable in the worst case. The bounds are tight in the number of states and actions ($S$ and $A$), providing the first rigorous sample complexity guarantees for recursive ERM across both risk-averse and risk-seeking regimes.
URL: https://openreview.net/forum?id=TFwSG4uYwl
---
Title: A New First-Order Meta-Learning Algorithm with Convergence Guarantees
Authors: El Mahdi Chayti, Martin Jaggi
Abstract: Learning new tasks by leveraging prior experience is a fundamental trait of intelligent systems. While Model-Agnostic Meta-Learning (MAML) is a leading approach, it suffers from significant computational and memory overhead due to the requirement of computing second-order meta-gradients. We propose \textbf{FO-B-MAML}, a novel first-order variant of MAML derived from a bi-level optimization perspective. Our framework introduces a new expression of the meta-gradient, defined as the derivative of the solution of a perturbed optimization problem. This formulation allows the meta-gradient to be estimated using various finite difference methods; in this work, we propose and analyze two simple yet effective estimators: a forward and a symmetric approximation.
Unlike existing first-order methods like FO-MAML and Reptile, which suffer from irreducible bias, we prove that FO-B-MAML converges to a stationary point of the meta-objective. Notably, the symmetric estimator achieves an improved $\mathcal{O}(\delta^{2/3})$ bias rate, strictly enhancing previous first-order theory. Furthermore, we demonstrate that the MAML objective violates standard smoothness assumptions; we show instead that its smoothness constant grows with the norm of the meta-gradient. This property theoretically justifies the use of normalized or clipped-gradient methods (SNGDM) over vanilla gradient descent.
Our empirical results validate these advancements: FO-B-MAML achieves high accuracy, closely following second-order MAML performance. Crucially, our method bypasses the ``activation bottleneck'' of second-order approaches, maintaining a flat memory footprint even when scaling to deep, activation-heavy CNNs and Transformers.
URL: https://openreview.net/forum?id=4zGAXQxisB
---
Title: Medix: Out-of-Distribution Detection from Unlabeled Wild Data via Robust Gradient Statistics
Authors: Momin Abbas, Ali Falahati, Hossein Goli, Mohammad Mohammadi Amiri
Abstract: Out-of-distribution (OOD) detection plays a crucial role in ensuring the robustness of machine learning systems deployed in real-world applications. Recent approaches have explored the use of unlabeled data, showing potential for enhancing OOD detection capabilities. However, effectively utilizing unlabeled in-the-wild data remains challenging due to the mixed nature of both in-distribution (InD) and OOD samples. The lack of a distinct set of OOD samples complicates the task of training an optimal OOD classifier. In this work, we introduce Medix, a novel framework designed to identify potential outliers from unlabeled data using the median-based robust gradient statistics. We use the median because it provides a stable estimate of the central tendency, as an OOD detection mechanism, due to its robustness against noise and outliers. Using these identified outliers, along with labeled InD data, we train a robust OOD classifier. From a theoretical perspective, we derive error bounds that demonstrate Medix achieves a low error rate. Empirical results further substantiate our claims, as Medix outperforms existing methods across the board in open-world settings.
URL: https://openreview.net/forum?id=jFjA24PBJx
---
Title: TOAM-YOLO: A Tiny Object-Aware Multi-Expert YOLO Framework for Diverse Domains
Authors: Vaibhav Sharma, Arnesh Batra, Arush Gumber, Ritu Gupta, Anubha Gupta
Abstract: YOLO-based object detection models have advanced significantly over the years through continuous architectural refinements and subsequent performance improvements. Tiny object detection remains a challenging task due to several constraints posed by progressive downsampling in model architectures and the smaller footprint of tiny objects in high-resolution images. This challenge is faced in diverse applications such as maritime surveillance, aerial surveillance, and in medical applications such as microscopic blood cell analysis. In this study, we introduce and demonstrate that a novel multi-domain expert, which we refer to as TOA-MoE (Tiny object aware mixture of experts), consisting of a Hessian-based curvature expert and a Fourier-based frequency expert, along with a 3-level attention mechanism, substantially improves the detection performance of YOLO models while only increasing the learnable parameters by a small fraction. Additionally, we add a high-resolution P2-level detection pathway integrated with a feature fusion network that incorporates a BiFPN-style structure and integrates deformable convolutional layer modules in the architecture, and we replace the standard up-sampling layers with a Content-Aware Reassembly of Features (CARAFE) module to preserve fine-grained feature details during feature map expansion. We systematically demonstrate the plug-and-play capability of these changes on YOLOv11 and YOLOv12 models. Tiny object aware Mixture of experts-based YOLO (TOAM-YOLO) achieves consistent improvements across five datasets: three tiny object benchmarks (SeaPerson, TinyPerson, VisDrone) with mAP@0.5 improvements of 11.6\%, 4.64\%, and 11\% respectively, and two blood cell datasets (BCCD, CBC) with mAP@0.5:0.95 improvements of 3.8\% and 1.7\% for platelet detection, all over YOLOv12n, while adding only 0.75M parameters. Ablation studies show that the high-resolution P2-level detection pathway provides the largest performance gain, with TOA-MoE providing an additional improvement on top.
URL: https://openreview.net/forum?id=2lIE1tgmRN
---
Title: Using RKHS Weight Functions in Random Feature Models
Authors: Gabriel Dubé, Mario Marchand, Frédéric LeBlanc
Abstract: We examine the consequences of positing that the weight function $\alpha$ in the classical random feature model formulation $f(x) = \E_{w\sim p}\qty[\alpha(w)\phi(w,x)]$ belongs to a reproducing kernel Hilbert space.
Depending on the choices of parameters of the random feature model, this assumption grants the ability to exactly calculate the model instead of relying on the random kitchen sinks method of approximation.
We present several such examples.
Additionally, using this form of the model, the functional gradient of the loss can be approximated in an unbiased way through sampling of the random features.
This allows using a stochastic functional gradient descent to learn the weight function.
We show that convergence is guaranteed under mild assumptions.
Further theoretical analysis shows that the empirical risk minimizer converges with the same $\Ocal\qty(\frac 1 {\sqrt m} + \frac 1 {\sqrt T})$ rate as Rahimi & Recht (2009).
We also present two other algorithms for learning the weight function.
We run experiments to compare these three learning algorithms, and to compare this random feature model variant to the original random kitchen sinks and other state of the art algorithms.
URL: https://openreview.net/forum?id=BCNPJ3WRVI
---
Title: OmegAMP: Targeted AMP Discovery via Biologically Informed Generation
Authors: Diogo Soares, Leon Hetzel, Paulina Szymczak, Marcelo Der Torossian Torres, Johanna Sommer, Cesar de la Fuente-Nunez, Fabian J Theis, Stephan Günnemann, Ewa Szczurek
Abstract: Deep learning-based antimicrobial peptide (AMP) discovery faces critical challenges such as limited controllability, lack of representations that efficiently model antimicrobial properties, and low experimental hit rates. To address these challenges, we introduce OmegAMP, a framework designed for reliable AMP generation with increased controllability. Its diffusion-based generative model leverages a novel conditioning mechanism to achieve fine-grained control over desired physicochemical properties and to direct generation towards specific activity profiles, including species-specific effectiveness. This is further enhanced by a biologically informed encoding space that significantly improves overall generative performance. Complementing these generative capabilities, OmegAMP leverages a novel synthetic data augmentation strategy to train classifiers for AMP filtering, drastically reducing false positive rates and thereby increasing the likelihood of experimental success. Our in silico experiments demonstrate that OmegAMP delivers state-of-the-art performance across key stages of the AMP discovery pipeline, enabling an unprecedented success rate in in vitro experiments. We tested 25 candidate peptides, 24 of them (96%) demonstrated antimicrobial activity, proving effective even against multi-drug resistant strains. Our findings underscore OmegAMP's potential to significantly advance computational frameworks in the fight against antimicrobial resistance.
URL: https://openreview.net/forum?id=hAq3XLZ9ex
---
Title: AuxiLight: Fast, Lightweight Auxiliary Loss Balancing Algorithm with Application in 6D Pose Estimation
Authors: Tianzong Zhang, Yinlin Hu, Kaicheng Yu, Mathieu Salzmann
Abstract: Jointly learning multiple tasks has proven to be beneficial. Auxiliary task learning builds on this by jointly training a set of predefined auxiliary tasks to improve the performance of a desired main task. Unfortunately, auxiliary task learning is often not practical because 1) jointly training multiple tasks requires an exponential hyperparameter search space of task loss weights; 2) the auxiliary tasks often lead to expensive annotation costs to obtain ground truth. In this work, we propose AuxiLight, a generic auxiliary learning algorithm, and consider the concrete real-world use case of 6D pose estimation. AuxiLight addresses the first issue via an algorithm that adaptively balances the auxiliary task losses based on an analysis of the training dynamics. Experiments on standard multi-task datasets show that our method consistently outperforms single-task models and state-of-the-art auxiliary task learning methods, being the fastest and the most lightweight among the known task-weighting algorithms. To demonstrate the practicality of auxiliary learning on real-world tasks, we further apply our method to 6D object pose estimation. We highlight that, for this task, multiple ground-truth auxiliary annotations can in fact be generated for free. This lets us showcase a concrete use of auxiliary learning for real-world problems that does induce annotation costs.
URL: https://openreview.net/forum?id=iQGmKWqr4r
---
Title: Retrievit: In-context Retrieval Capabilities of Transformers, State Space Models, and Hybrid Architectures
Authors: Georgios Pantazopoulos, Malvina Nikandrou, Ioannis Konstas, Alessandro Suglia
Abstract: Transformers excel at in-context retrieval but suffer from quadratic complexity with sequence length, while State Space Models (SSMs) offer efficient linear-time processing but have limited retrieval capabilities. We investigate whether hybrid architectures combining Transformers and SSMs can achieve the best of both worlds on two synthetic in-context retrieval tasks. The first task, n-gram retrieval, requires the model to identify and reproduce an n-gram that succeeds the query within the input sequence. The second task, position retrieval, presents the model with a single query token and requires it to perform a two-hop associative lookup: first locating the corresponding element in the sequence, and then outputting its positional index. Under controlled experimental conditions, we assess data efficiency, length generalization, robustness to out of domain training examples, and learned representations across Transformers, SSMs, and hybrid architectures. We find that hybrid models outperform SSMs and match or exceed Transformers in terms of data efficiency and extrapolation for tasks that require precise information retrieval from the input context.However, Transformers maintain superiority in position retrieval tasks. Through representation analysis, we discover that SSM-based models develop locality-aware embeddings where tokens representing adjacent positions become neighbors in embedding space, forming interpretable structures. This property is absent in Transformers as causal attention is sufficient for acquiring positional associations, and the introduction of positional encoding amplifies this behavior, leading to a consequent improvement in data efficiency. SSMs on the other hand update their internal representations incrementally and without positional encodings, are required to learn these associations. Our findings reveal fundamental differences in how Transformers and SSMs, and hybrid models learn positional associations.
URL: https://openreview.net/forum?id=PxkhbVeRRj
---
Title: Flex-Act: Why Learn when you can Pick?
Authors: Ramnath Kumar, Kyle Ritscher, Junmin Zhu, Lawrence Liu, Cho-Jui Hsieh
Abstract: Learning activation functions has emerged as a promising direction in deep learning, allowing networks to adapt activation mechanisms to task-specific demands. In this work, we introduce a novel framework that employs the Gumbel-Softmax trick to enable discrete yet differentiable selection among a predefined set of activation functions during training. Our method dynamically learns the optimal activation function independently of the input, thereby enhancing both predictive accuracy and architectural flexibility. Experiments on synthetic datasets show that our model consistently selects the most suitable activation function, underscoring its effectiveness. These results connect theoretical advances with practical utility, paving the way for more adaptive and modular neural architectures in complex learning scenarios.
URL: https://openreview.net/forum?id=HQGis83pM2
---
Title: Privacy-Aware Visual Language Models
Authors: Laurens Samson, Nimrod Barazani, Sennay Ghebreab, Yuki M Asano
Abstract: As Visual Language Models (VLMs) become increasingly embedded in everyday applications. Ensuring they can recognise and appropriately handle privacy-sensitive content is thus essential to protect users. To this end, we conduct a comprehensive evaluation of ten state-of-the-art VLMs and identify limitations in their understanding of visual privacy. However, existing privacy-related datasets often suffer from label inconsistencies, limiting their reliability. To address this, we introduce two compact, high-quality benchmarks, PrivBench and PrivBench-H, that focus on commonly recognised visual privacy categories aligned with the General Data Protection Regulation (GDPR). Additionally, we present PrivTune, an instruction-tuning dataset specifically curated to improve privacy sensitivity. We obtain multiple Privacy VLMs by fine-tuning off-the-shelf VLMs on only 100 samples from PrivTune, which leads to substantial gains on all benchmarks, surpassing even GPT-4, while maintaining strong performance on other tasks. Our findings show that privacy-awareness in VLMs can be substantially improved with minimal data and careful dataset design, setting the stage for safer, more privacy-aligned AI systems.
URL: https://openreview.net/forum?id=ntLnq05sBA
---
Title: VBased: Revisiting Architectural Design for Simple Linear Attention Models in Vision
Authors: Yibo Zhong
Abstract: Linear attention has been proposed as an efficient alternative to softmax attention, particularly for language modeling. Motivated by its lower computational complexity, many works have applied linear attention to vision tasks as well. In this paper, we focus on simple isotropic architectures and investigate two key architectural design aspects of extending linear attention to visual data: scanning methods and hybrid architectures combining linear and softmax attention. We study a series of scanning methods, and empirical results suggest that the scanning strategy itself provides limited benefits. In contrast, hybrid models yield promising results compared to pure linear attention models. Focusing on the hybrid design, we further investigate several types of softmax attention suitable for integration and find that the tiled version of high-order sliding window attention (HSWA) is efficient in both theory and practice. We name the resulting simple architecture, which combines linear attention with HSWA, VBased, and conduct additional experiments to evaluate its effectiveness. With performance comparable to Transformers and equal efficiency (on moderate sequences) or superior efficiency (on long sequences), VBased offers a promising path for the adoption of linear attention in vision and can serve as a simple baseline for future architectural research.
URL: https://openreview.net/forum?id=bMDc1wMkNe
---
Title: Closing the Modality Gap for Mixed Modality Search
Authors: Binxu Li, Yuhui Zhang, Xiaohan Wang, Weixin Liang, Ludwig Schmidt, Serena Yeung-Levy
Abstract: Mixed modality search, retrieving information across a heterogeneous corpus composed of images, texts, and multimodal documents, is an important yet underexplored real-world application. In this work, we investigate how contrastive vision-language models, such as CLIP, perform on the mixed modality search task. Our analysis reveals a critical limitation: these models exhibit a pronounced modality gap in the embedding space, where image and text embeddings form distinct clusters, leading to intra-modal ranking bias and inter-modal fusion failure. To address this issue, we propose GR-CLIP, a lightweight post-hoc calibration method that removes the modality gap in CLIP’s embedding space. Evaluated on MixBench, the first benchmark specifically designed for mixed modality search, GR-CLIP improves NDCG@10 by up to 26% over CLIP, surpasses recent vision-language generative embedding models by 4%, while using 75x less compute.
URL: https://openreview.net/forum?id=jKy1yDMFcs
---
Title: Self-Consistent Flow: Unifying Velocity and Endpoint Prediction for Rectified Flow Models
Authors: Xu Han, Jiajing Hu, Liping Liu
Abstract: In rectified-flow–based generative models, the neural network can be trained to predict two different targets, such as the instantaneous velocity or the data endpoint, to perform denoising. Although prior work shows that these parameterizations lead to different empirical behaviors, the mechanisms underlying their respective advantages remain to be underexplored, and how to combine them effectively is still unclear. In this work, we analyze how learning errors from different parameterizations affect the generation performance. We show that predicting the data endpoint has a clear training signal that stabilizes training, whereas predicting the velocity maintains stable sampling dynamics near the data manifold. Motivated by these insights, we propose Self-Consistent Flow (SC-Flow), a new method that unifies the benefits of both parameterizations. By employing a lightweight consistency loss, SC-Flow jointly trains a single network to predict both the local velocity and the data endpoint, and the consistency between the two predictions improves the model's performance. The method requires no major architectural changes and adds minimal computational overhead. Extensive experiments on image generation tasks demonstrate that SC-Flow substantially stabilizes optimization and improves the straightness of generation paths, leading to significant gains in generation quality over standard rectified-flow baselines.
URL: https://openreview.net/forum?id=z9FMXOJgyl
---
Title: Causally Fair Node Classification on Non-IID Graph Data
Authors: Yucong Dai, Lu Zhang, Yaowei Hu, Susan Gauch, Yongkai Wu
Abstract: Fair machine learning seeks to identify and mitigate biases in predictions against unfavorable populations characterized by demographic attributes, such as race and gender. Recent research has extended fairness to graph data, such as social networks, but many studies neglect the causal relationships among data instances. This paper addresses a prevalent challenge in many fair machine learning research, which typically assumes independent and identically distributed (IID) data, from the causal perspective. Specifically, this work targets the circumstance where nodes with different neighborhood structures follow different causal mechanisms, violating the invariance assumptions required for classical structural causal models and $do$-calculus. We base our research on the Network Structural Causal Model (NSCM) framework and develop a Message Passing Variational Autoencoder for Causal Inference (MPVA) to compute interventional distributions for causally fair node classification. We establish theoretical soundness under two conditions: Decomposability and Graph Independence. These conditions formalize when causal mechanism heterogeneity can be overcome by constructing a structural representation that restores invariance and facilitates the computation of interventional distributions using $do$-calculus in non-IID settings. Empirical evaluations on semi-synthetic and real-world datasets demonstrate that MPVA outperforms conventional methods by effectively approximating interventional distributions and mitigating bias. Our findings demonstrate the potential of causality-based fairness in complex ML applications and motivate future work on relaxing the classic assumptions in algorithmic fairness.
URL: https://openreview.net/forum?id=AwptwzGld5
---
New submissions
===============
Title: Recursive Entropic Variational Inference for Nonlinear State-Space Models
Abstract: We present a class of algorithms for state estimation in nonlinear, non-Gaussian state-space models. Our approach is based on a variational Lagrangian formulation that casts Bayesian inference as a sequence of entropic trust-region updates subject to dynamic consistency constraints. This framework gives rise to a family of forward-backward algorithms whose structure is determined by the chosen factorization of the variational posterior. By focusing on Gauss--Markov approximations, we derive recursive schemes with favorable computational complexity. For general nonlinear, non-Gaussian models, we close the recursions using generalized statistical linear regression and Fourier--Hermite moment matching.
URL: https://openreview.net/forum?id=Pt3zdEdg5y
---
Title: Robustness Rankings Sort on the Wrong Axis: The Case for Iso-Depth Evaluation in Machine Unlearning
Abstract: Unlearning methods are ranked by their resistance to relearning: how quickly fine-tuning recovers a forgotten set. We show these rankings are confounded by suppression depth, how far into the forget loss a method drives the model, which single-run, default-recipe comparisons leave uncontrolled. On a depth-controlled ladder across three model families (GPT-2-medium, Phi-1.5, Llama-3.2-1B; Llama-3.2-3B) and two benchmarks (TOFU, MUSE), a balanced variance decomposition attributes relearning resistance overwhelmingly to depth and run-to-run noise on TOFU: depth explains 63--68\% of the variance over our ladder and the method only 0.5--8.1\%, below run-noise in every depth window. We propose iso-depth evaluation: compare at matched depth, report run variance, and pin the scored checkpoint, rather than ranking default recipes. We observed that the residual method effect is small on TOFU but larger on MUSE (28--37\%), tracks a second measurable quantity, weight displacement: at matched depth, resistance covaries with displacement and the spread of displacement across methods tracks the benchmark-dependent method share. This second axis is observational; within fictitious-QA it is secondary to depth, while on the corpus it is the larger within-benchmark predictor. We report it as a refinement of the depth result, not a co-equal controlled axis.
URL: https://openreview.net/forum?id=QhJ3xDwAC3
---
Title: HOGT: High-Order Graph Transformers for Graph Representation Learning
Abstract: Inspired by the success of transformers in natural language processing (NLP) and computer vision (CV), graph transformers (GTs) have recently been proposed to improve graph learning performance. However, existing GTs struggle to capture essential topological structures or scale to large graphs due to their quadratic computational complexity.
To address these limitations, we propose a high-order information propagation strategy within the transformer architecture to jointly model local, long-range, and higher-order relationships in graphs. We first propose flexible sampling method to extract communities from the graph, and create new community nodes and in particular a learnable community sampling method with reinforcement learning. We then propose a three-step message-passing strategy dubbed HOGT to capture the local and higher-order information in the communities and propagate long-range dependency information between the community nodes to finally obtain comprehensive node representations.
Notably, by explicitly integrating structural information into our community-based message-passing, HOGT discards the positional encoding which was thought to be important for GT. We theoretically demonstrate that GTs with effective substructures can achieve an approximate global attention. HOGT can be viewed as a unified framework, taking many existing graph models as its special cases. We empirically show that HOGT achieves highly competitive results consistently across node and graph classification, and link prediction tasks.
URL: https://openreview.net/forum?id=hxO7adfYhl
---
Title: EC-LoRA: Energy-Driven Continual LoRA Implicit Generation
Abstract: Low-Rank Adaptation (LoRA) enables lightweight adaptation of large language models (LLMs), but continual adaptation across tasks still requires maintaining and selecting task-specific adapters. To address this issue, we propose EC-LoRA, an energy-guided implicit generative framework for producing LoRA modules in a continual learning setting. Instead of directly decoding adapter weights with an explicit generator, EC-LoRA learns a task-conditioned energy function over the normalized parameter space. For each task, we initiate a prototype and generate a task-specific adapter by refining via energy-based gradient descent. The energy model is trained with score matching and updated sequentially across tasks. Our method generates only LoRA and reuses the non-LoRA classification or regression head from a selected source checkpoint, making it applicable to tasks with heterogeneous output spaces. We conducted experiments to verify that the LoRA generated by EC-LoRA outperforms the original LoRA and can adapt to different models and scenarios.
URL: https://openreview.net/forum?id=LfyISGU7Ul
---
Title: Exploring the Rashomon Set for Concept-Based Models
Abstract: In many machine learning problems, there may exist multiple models that achieve nearly identical predictive performance while relying on fundamentally different internal logic. However, standard training procedures produce a single model, offering no practical way to explore alternatives that may better suit downstream needs. The set of these equally accurate models is known as the Rashomon set. Exploring the Rashomon set is particularly challenging in large and complex hypothesis spaces, such as Concept Bottleneck Models (CBMs), which are widely used in computer vision to make predictions through intermediate, human-understandable concepts. In this paper, we provide a method for efficiently exploring the Rashomon set of CBMs. Our framework introduces a specialized parallel adapter-based construction, combined with a checkpointing scheme and a concept diversity objective, to generate multiple equally accurate CBMs from a single training process. Empirical results show that our method finds models with better diversity than baselines while using much less memory. We further demonstrate that access to these diverse yet accurate CBMs enables trustworthy model selection, resolution of inter-class confusion, and reliable abstention in decision-making.
URL: https://openreview.net/forum?id=E8oetPdxH6
---
Title: Quantifying Uncertainty in LLM Descriptions of Binary Code Functions
Abstract: Large language models have demonstrated effectiveness in summarizing source code, suggesting potential in using them to summarize compiled software binaries. However, compilation removes linguistic cues used for comprehension, leading to a lower information environment with greater reliability challenges than source code. Uncertainty quantification (UQ) methods offer a way to identify low-quality summaries but have not yet been explored in this domain. We create a framework generating descriptions of functions from decompiled software, score them against ground truth summaries created from source code, and evaluate the ability of UQ metrics to separate correct and incorrect descriptions. Our results show several metrics capable of identifying functions with unreliable description quality, with the best performing metric using cosine similarities between embeddings to measure the consistency of descriptions generated by leveraging input variations of related clusters in the call graph, demonstrating UQ's potential to validate results to users in this domain.
URL: https://openreview.net/forum?id=vWtGZS1bvM
---
Title: Gaussian Process-based learning with new MCMC-based implementation of Wishart prior on correlation matrix
Abstract: In probabilistic supervised learning of an input-output relationship - as a sample function of a Gaussian Process (GP) - priors are typically specified for the hyperparameters of the kernel that parametrises the covariance function of the GP, where the induced covariance matrix of the (resulting multivariate Normal) likelihood, governs the learning and prediction. When the sought function is highly multivariate, multiple lengthscale parameters must be learnt simultaneously, making inference difficult. We develop a ``self-assembled'' Wishart prior for the covariance matrix, while undertaking Bayesian inference on the kernel hyperparameters using MCMC. The construction uses a look-back window over recent MCMC iterations to define a time-step dependent scale matrix, thereby introducing adaptiveness to the chain. Results suggest that direct prior specification on the covariance matrix can be useful for diagnosing weakly informative inputs within the GP-based learning paradigm. We support our prior development with two distinct empirical illustrations - one on synthetic data, and another on a real-world dataset.
URL: https://openreview.net/forum?id=aLkrVKUOys
---
Title: BID-LoRA: A Parameter-Efficient Framework for Continual Learning and Unlearning
Abstract: Recent advances in deep learning underscore the need for systems that can not only acquire new knowledge through Continual Learning (CL) but also remove outdated, sensitive, or private information through Machine Unlearning (MU). However, while CL methods are well-developed, MU techniques remain in early stages, creating a critical gap for unified frameworks that depend on both capabilities. We find that naively combining existing CL and MU approaches results in knowledge leakage a gradual degradation of foundational knowledge across repeated adaptation cycles. To address this, we formalize Continual Learning Unlearning (CLU) as a unified paradigm with three key goals: (i) precise deletion of unwanted knowledge, (ii) efficient integration of new knowledge while preserving prior information, and (iii) minimizing knowledge leakage across cycles. We propose Bi-Directional Low-Rank Adaptation (BID-LoRA), a novel framework featuring three dedicated adapter pathways-retain, new, and unlearn-applied to attention layers, combined with escape unlearning that pushes forget-class embeddings to positions maximally distant from retained knowledge, updating only ≈5% of parameters. Experiments on CIFAR-100 show that BID-LoRA outperforms CLU baselines across multiple adaptation cycles. We further evaluate on CASIA-Face100, a curated face recognition subset, demonstrating practical applicability to real-world identity management systems where new users must be enrolled and withdrawn users removed.
URL: https://openreview.net/forum?id=xkt4R2bx9O
---
Title: Tackling the Noisy Elephant in the Room: Label Noise-robust Out-of-Distribution Detection via Loss Correction and Low-rank Decomposition
Abstract: Robust out-of-distribution (OOD) detection is an indispensable component
of modern artificial intelligence (AI) systems, especially in
safety-critical applications where models must identify inputs from
unfamiliar classes not seen during training. While OOD detection has been
extensively studied---with both post hoc and training-based
approaches---its effectiveness under noisy training labels remains
underexplored. Recent studies suggest that label noise can significantly
degrade OOD performance, yet principled solutions are lacking.
In this work, we demonstrate that directly combining existing label
noise-robust methods with OOD detection strategies is insufficient to
address this critical challenge. To overcome this, we propose a
\textit{robust} OOD detection framework designed to \textit{cleanse}
feature embeddings, thereby mitigating the adverse effects of noisy labels
on OOD performance. Towards this, we introduce an end-to-end training
strategy that integrates loss correction methods from the noisy-label
learning literature with low-rank and sparse decomposition techniques from
signal processing. Building on this strategy, we derive a novel metric
that quantifies the ``OOD-ness'' content within training data, which in
turn leads to a label noise-robust OOD detection scoring technique.
Extensive experiments on both synthetic and real-world datasets demonstrate
that our method significantly outperforms the state-of-the-art OOD
detection techniques, particularly under severe noisy label settings.
URL: https://openreview.net/forum?id=xHRsY50dVt
---
Title: Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond
Abstract: As AI systems move from generating text to accomplishing goals through sustained interaction, the ability to model environment dynamics becomes a central bottleneck. Agents that manipulate objects, navigate software, coordinate with others, or design experiments require predictive environment models, yet the term world model carries different meanings across research communities. We introduce a "levels × laws" taxonomy organized along two axes. The first defines three capability levels: L1 Predictor, which learns one-step local transition operators; L2 Simulator, which composes them into multi-step, action-conditioned rollouts that respect domain laws; and L3 Evolver, which autonomously revises its own model when predictions fail against new evidence. The second identifies four governing-law regimes (physical, digital, social, and scientific) that determine what constraints a world model must satisfy and where it is most likely to fail. Using this framework, we synthesize over 400 works and summarize more than 100 representative systems spanning model-based reinforcement learning, video generation, web and GUI agents, multi-agent social simulation, and AI-driven scientific discovery. We analyze methods, failure modes, and evaluation practices across level-regime pairs, propose decision-centric evaluation principles and a minimal reproducible evaluation package, and outline architectural guidance, open problems, and governance challenges. The resulting roadmap connects previously isolated communities and charts a path from passive next-step prediction toward world models that can simulate, and ultimately reshape, the environments in which agents operate.
URL: https://openreview.net/forum?id=WQyoyqDQ1D
---
Title: Don't Give Up Fairness Even in the Midst of Membership Inference Attack
Abstract: Existing membership privacy-defending algorithms have demonstrated effectiveness in protecting neural networks against a wide range of membership inference attacks (MIAs). However, we observed that these privacy defense training algorithms often exacerbate disparities in predictive performance across data groups. In detail, existing privacy-defending techniques tend to sacrifice the generalizability of minority classes/groups disproportionately, while largely preserving the performance of majority groups. To address this issue, we propose Fair and Privacy-Safe Tuning (FaPriST) to mitigate unfairness while maintaining strong privacy preservation. Extensive evaluations show that our approach consistently outperforms existing privacy-defending approaches, even with no defense model, in terms of fairness, while achieving comparable robustness against membership inference attacks.
URL: https://openreview.net/forum?id=mQzedmdOdX
---
Title: Building LLM Agent Systems the Deep Learning Way: From Modular Design to Architecture Search
Abstract: Large Language Models (LLMs) have revolutionized AI research and enabled exciting agent
systems. To build a complex LLM agent system, most existing research relies on insights
from other domains or heuristics to manually build the agent system. However, this approach
often requires heavy hand-engineering and fails to fully optimize for the downstream task of
interest. Inspired by the tremendous success of deep learning, we propose to construct LLM
agent systems in a modular manner, similar to building a deep neural network. Our key
insight is to make analogies between LLM building blocks, such as retrievals, memories, and
prompting strategies, and the successful deep learning modules, such as MLPs, attention,
and recurrent modules. We further design forward inference and feedback mechanisms
for LLMs, where prompts in LLMs are considered as the weights in deep models, and
the prompt optimization from feedback is analogous to the back-propagation algorithm.
We additionally leverage a search algorithm to search for the best configuration of LLM
agent systems, similar to the neural architecture search (NAS) in deep learning research.
Comprehensive experimental results demonstrate that the proposed deep learning recipe
for LLM agent systems is highly effective, in particular: (1) Organizing LLM modules into
deep-learning-style architectures yields noticeable performance gain; (2) Automatic prompt
optimization, equivalent to backpropagation, is efficient in incorporating feedback from the
task of interest and achieves at least 5% performance improvement; (3) NAS equivalent
algorithm works well for further optimizing the LLM agent system architecture with 11%
performance gain compared with randomly designed architectures. Overall, our research
demonstrates the exciting opportunity of transferring the success of deep learning to building
LLM agent systems.
URL: https://openreview.net/forum?id=PS5Cm4zwCj
---
Title: LRanker: LLM Ranker for Massive Candidates
Abstract: Large language models (LLMs) have recently shown strong potential for ranking by capturing semantic relevance and adapting across diverse domains, yet existing methods remain constrained by limited context length and high computational costs, restricting their applicability to real-world scenarios where candidate pools often scale to millions. To address this challenge, we propose LRanker, a framework tailored for large-candidate ranking. LRanker incorporates a candidate aggregation encoder that leverages K-means clustering to explicitly model global candidate information, and a graph-based test-time scaling mechanism that partitions candidates into subsets, generates multiple query embeddings, and integrates them through an ensemble procedure. By aggregating diverse embeddings instead of relying on a single representation, this mechanism enhances robustness and expressiveness, leading to more accurate ranking over massive candidate pools. We evaluate LRanker on seven tasks across three scenarios in RBench with different candidate scales. Experimental results show that LRanker achieves over 30% gains in the RBench-Small scenario, improves by 3–9% in MRR in the RBench-Large scenario, and sustains scalability with 20–30% improvements in the RBench-Ultra scenario with more than 6.8M candidates. Ablation studies further verify the effectiveness of its key components. Together, these findings demonstrate the robustness, scalability, and effectiveness of LRanker for massive-candidate ranking.
URL: https://openreview.net/forum?id=9VBpOwbXrZ
---
Title: NVP: Neural Vitality based Dynamic Sparsity Pruning in Convolutional Neural Networks
Abstract: As neural networks grow larger, improving computational efficiency without sacrificing accuracy is increasingly important for sustainable deep learning. We study this problem through the lens of dying neurons, whose activations show low variability and limited expressiveness. We introduce neuron entropy, a data-dependent measure of activation variability that quantifies each neuron’s contribution to network expressiveness. Combining this measure with weight magnitude, we further define the Neural Vitality Index (NVI) to assess neuron-level importance. Based on NVI, we propose Neural Vitality Pruning (NVP), a dynamic pruning framework that adaptively sparsifies convolutional neural networks during training by removing low-vitality neurons. Experiments across representative CNN architectures and competitive pruning baselines show that NVP achieves higher sparsity while maintaining or improving predictive performance. These results suggest neuron vitality as a principled criterion for efficient neural network optimization.
URL: https://openreview.net/forum?id=do4AjOggmN
---
Title: Langevin Monte-Carlo Provably Learns Depth Two Neural Nets at Any Size and Data
Abstract: In this note we prove that the Langevin Monte Carlo (LMC) algorithm can learn depth-$2$ neural nets of any size and for any data --- and we give non-asymptotic convergence rates for it. We achieve this via obtaining distributional convergence of iterates for the stochastic training algorithm, LMC, on nets of any size. In particular, we show that in q-R\'enyi\xspace divergence, the iterates of LMC converge to the Gibbs distribution of certain Frobenius norm regularized losses for these nets, in both classification and regression settings. The amount of regularization needed for our results is independent of the size of the net. This work synthesizes several recent observations about isoperimetry conditions under which LMC converges and that two-layer neural loss functions can always be regularized by a certain constant amount such that they satisfy the Villani conditions, and thus their Gibbs measures satisfy a Poincar\'e\xspace inequality.
URL: https://openreview.net/forum?id=QkVwGr7sRR
---
Title: Identifiability and Recovery of Episodic Coordination in Switching Marked Point Processes
Abstract: We study when episodic coordination is identifiable and recoverable in regime-switching marked Hawkes processes, where a brief coupled episode is easily confused with burstiness, time-averaged coupling, or segmentation artifacts. We show recovery requires four ingredients: sufficient effective active exposure, a source of dormant--active separation, birth-time attribution of excitation, and a normalization separating actor couplings from mark-transition weights. A finite-sample analysis identifies birth-time active-born events from true actors as the effective sample size and, under an explicit posterior-accuracy condition, transfers standard sparse support-recovery machinery to this setting, with weak links requiring quadratically more exposure. These conditions motivate a birth-time gated marked Hawkes estimator with role-separated sender/receiver gates, asymmetric low-rank directional structure, and generalized-EM activation recovery. Synthetic experiments, including an external change-point Hawkes baseline, internal identifiability ablations, sparse directed-edge diagnostics, and posterior-accuracy stress tests, validate the exposure threshold, support the posterior-accuracy bridge used by the recovery theorem, and show the predicted collapse when separation conditions are violated. A dedicated synthetic weakest-link sweep confirms the predicted difficulty ordering where weaker links require longer active exposure. Empirically, planted-coordination and predictive checks show that the estimator recovers member and receiver sets on volume-neutral UCDP GED conflict-event data, a cross-asset mark-transition channel on FTX-collapse trade-event data, and the same recovery ordering on real hippocampal CA1 recordings, while exposure, separation, or attribution failures lead to non-identifiability or practical recovery failure.
URL: https://openreview.net/forum?id=xrGM4Z5I2h
---
Title: Inference-Time Reward Alignment of Diffusion Models via Geometric Particle Expansion
Abstract: Diffusion models have demonstrated strong generative performance in diverse domains, including images, molecular structures, and scientific design. Aligning pre-trained diffusion models with domain-specific objectives at inference time, without fine-tuning, has recently gained attention, particularly in settings involving non-differentiable rewards. Inspired by recent advances in Sequential Monte Carlo sampling, we propose a derivative-free alignment framework for diffusion model inference. We introduce a geometric expansion within Sequential Monte Carlo-based sampling that avoids the need for reward gradients for guidance during inference. Our approach generates additional particles via geometric noise derived from the covariance of elite particles along dominant eigendirections, followed by resampling at the same timestep to mitigate elite particle degeneracy. We demonstrate that our method consistently generates designs with improved performance and diversity when optimizing non-differentiable rewards across image, molecules, and DNA sequence generation tasks, while achieving comparable or superior reward scores for differentiable aesthetic score, without requiring reward-gradient computation.
URL: https://openreview.net/forum?id=mAs42AIhS3
---
Title: The Points You Didn’t See: Continuous Domain Adaptation with Koopman Operator and Gaussian Mixture Optimal Transport
Abstract: In this paper, we introduce a continuous domain adaptation method that learns domain-invariant representations by connecting optimal transport and transfer operators theory. We represent empirical source and target samples as Gaussian mixtures, solve an entropic GMM optimal transport problem between them, and use the resulting plan to construct a continuous weighted graph on which we define a Koopman operator. The key insight is that the infinite-dimensional Koopman eigenproblem admits an exact finite-dimensional reduction, yielding closed-form embeddings that we show are domain-invariant and enable out-of-sample mapping. Empirical validation on benchmarks spanning audio, signal processing, and visual adaptation modalities shows that our approach consistently outperforms OT-based, deep domain adaptation, and neural OT baselines, with a decisive advantage on out-of-sample generalization.
URL: https://openreview.net/forum?id=dZgWEnifsC
---
Title: Kuramoto Attention: Synchronizing Self-Attention on the Torus
Abstract: Transformer models are increasingly used as computational models of cognition and neural representation, so the mechanism implemented by self-attention is of interest beyond engineering performance. A complementary tradition in cognitive science models coordination, binding, and memory through dynamical interactions such as oscillator synchrony; we bring this mechanism into self-attention by introducing the Kuramoto Attention layer, whose value update is a synchronization step. Each token carries a bank of phase oscillators, so its hidden state lives on a high-dimensional torus. The attention weights form an adaptive coupling graph, and using the raw phase states as values makes the value update exactly the Kuramoto coupling direction for fixed attention weights. The softmax selects which oscillators couple, while the value path moves each token toward the attention-weighted circular mean of the tokens it selects. We train Kuramoto Attention on enwiki8 and CodeParrot against parameter-matched RoPE+SwiGLU transformers. At 5M parameters on CodeParrot, it improves on the transformer by both median and mean, with mean gaps of 0.012 validation and 0.010 test bits per byte. At 5M on enwiki8, all six runs have lower validation/test medians than the transformer and all-seed means within 0.01 BPC; five of six also form a tight lower-mean cluster. At 1M, it trails by about 0.02 BPC on enwiki8 and by 0.013-0.015 bits per byte on CodeParrot. Ablations and phase diagnostics show how the layer’s synchronization and geometry-motivated components shape model performance. The result is a self-attention mechanism whose learned computation can be read directly as adaptive synchronization on phase states.
URL: https://openreview.net/forum?id=FFQWVfhJPw
---
Title: Optimal Transport Meets Reinforcement Learning: A Survey
Abstract: Reinforcement learning (RL) algorithms frequently compare probability distributions: the states a policy visits versus those an expert visits, the actions a learned policy takes versus those in a dataset, or the transitions a model predicts versus those the environment produces. Standard divergences such as Kullback--Leibler and Jensen--Shannon can fail when these distributions overlap weakly—a common situation in imitation learning, offline RL, and deployment under distribution shift. Optimal transport (OT) offers an alternative by measuring the cost of \emph{moving} probability mass from one distribution to another under a ground cost that encodes task geometry. This often provides informative gradients even under support mismatch. This survey covers how OT is used inside RL objectives and algorithms. For each method, we identify: the role OT plays (reward signal, regulariser, constraint, ambiguity set, or model loss), the distributions compared (occupancies, action conditionals, trajectory embeddings, transition kernels, or return distributions), the OT formulation used ($\mathsf{W}_1$, $\mathsf{W}_2$, entropy-regularised/Sinkhorn, Gromov--Wasserstein, or partial/unbalanced variants), and the treatment of temporal structure. We summarise practical guidance on cost design, regularisation tuning, and numerical stability, and highlight open problems including scalable trajectory-level transport, principled handling of mass mismatch, and theoretical guarantees for OT-regularised RL.
URL: https://openreview.net/forum?id=yIYUFUXJuq
---
Title: APEX4: Efficient Pure W4A4 LLM Inference via Intra-SM Compute Rebalancing
Abstract: W4A4 quantization promises full utilization of INT4 Tensor Cores, yet group dequantization overhead on CUDA Cores has driven existing systems to mixed-precision fallbacks. We present the first systematic study of how intra-SM compute balance governs this bottleneck. Through controlled benchmarks across four GPUs from Ampere and Ada architectures, we identify the Tensor Cores to CUDA Cores throughput ratio ($\rho$) as the primary hardware indicator: the W4A4-g128 kernel yields $2.0$--$2.5\times$ speedup on RTX~3090 ($\rho=16$) yet degrades to $0.43$--$0.47\times$ on A100 ($\rho=64$) in compute-bond scenarios, establishing W4A4 viability as platform-dependent rather than universally infeasible.
Guided by this finding, we build \textbf{APEX4}, which co-designs pure INT4 GEMM kernels with $\rho$-aware granularity adaptation to mitigate the CUDA Cores dequantization bottleneck.
APEX4 achieves perplexity within 0.63 of FP16 on Llama-2-70B and outperforms W4Ax Atom-g128 by 4.0\%--4.4\% in zero-shot accuracy. Deployed as a drop-in replacement in unmodified vLLM, it delivers up to $1.66\times$ end-to-end speedup on L40S ($\rho=8$), and $1.78\times$ on RTX~3090 ($\rho=16$), $2.09\times$ on A40 ($\rho=16$), while recovering A100 ($\rho=64$) to $1.20$--$1.40\times$ via the mixed-granularity mode.
URL: https://openreview.net/forum?id=A3GPeESWAN
---
Title: RECAP: Reconstructing the Past to Predict the Future in Time-Series Foundation Models
Abstract: Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm for enhancing Time Series Foundation Models (TSFMs) with domain-specific historical knowledge, mitigating their limitations under distribution shifts and non-stationary dynamics. However, existing retrieval-based forecasting approaches typically rely on shallow similarity measures, such as fixed distance metrics or naive correlations, which often retrieve superficially similar patterns that are not truly predictive of future behavior. In this work, we propose a learnable coarse-to-fine retrieval framework centered on a simple principle: reconstructing the past to predict the future. Rather than retrieving examples based solely on surface similarity, our method learns to identify historical contexts whose latent temporal structure can best reconstruct the query sequence and whose future trajectories provide the most informative predictive signals. Specifically, we first perform efficient candidate selection using approximate nearest neighbor search, followed by a learned reranking stage operating in a shared feature space. The reranker evaluates retrieved segments according to how well a weighted combination of their representations can reconstruct the original query context, coupling retrieval directly to downstream forecasting quality. In this way, retrieval becomes not merely a similarity search problem, but a reconstruction-driven mechanism for discovering predictive temporal analogies. Importantly, the entire retrieval pipeline operates on top of a frozen TSFM backbone, incurring only some extra lightweight modules. We further introduce a wavelet-based multi-scale regularization loss on the median quantile prediction to improve temporal consistency across different frequency bands. Extensive experiments on public benchmark datasets demonstrate that our method consistently outperforms existing forecasting baselines and retrieval-based approaches.
URL: https://openreview.net/forum?id=zHEmblwwBW
---
Title: Sign Conflicts in LoRA Adapter Merging Are Pervasive, Not Localized
Abstract: LoRA adapters are small additive weight updates that specialize a large language model to a
new task without retraining it. Merging several independently trained adapters into a single
model is appealing, but it confronts a basic obstacle: at many weights the adapters disagree
about whether the value should move up or down, a disagreement known as a sign conflict.
The standard remedy, TIES-Merging, resolves these conflicts through a single global vote
over all weights. Whether such global treatment is necessary is an open question with prac-
tical stakes. If the conflicts were concentrated in a few identifiable regions of the network,
they could be resolved only there, far more cheaply, and several recent merging methods qui-
etly assume some such concentration. That assumption has never been tested directly. This
work presents a systematic empirical analysis of the optimization and structural behaviors
that govern parameter-efficient fine-tuning, providing new insights into how sign conflicts
and destructive interference manifest across transformer architectures. This work tests it on
five language models (Qwen2.5-3B, Llama-3.2-3B, Phi-3.5-mini, Mistral-7B, OLMo-2-1B),
each adapted to three sentiment domains with five random seeds, under two measurements:
the usual count of how many weights disagree, and a magnitude-weighted measure of how
much update the disagreements actually erase in a merge. Both measurements reject the
assumption. By count, conflict is pervasive and nearly uniform: roughly 73% of weights dis-
agree, close to the 75% that purely random signs would produce, and two adapters trained
on the same data under different seeds disagree as often as adapters trained on different
tasks. The conflict therefore originates in the randomness of training rather than in the
tasks diverging, and it exposes no pattern a method could exploit. Weighting by magnitude
does reveal a genuine pattern: most of the erased update lies in the MLP input projections,
which are also the largest adapted tensors, in all five models. Yet even this pattern is too
diffuse to act on, since capturing 80% of the damage still requires touching roughly 72%
of the weights. The implication for efficient merging is cautionary: the damage is real but
thinly spread, so resolving conflicts only in selected regions cannot replace the global vote.
URL: https://openreview.net/forum?id=ZzzXwoWJj0
---
Title: Revisiting Dual Policies with Bayesian Optimism in Reinforcement Learning
Abstract: Deep reinforcement learning algorithms for continuous control often rely on conservative Q-value estimates to stabilise actor-critic learning. This conservatism is useful for exploitation, but it can also discourage actions whose value is uncertain yet potentially high. Prior work has addressed exploration by adding intrinsic rewards, explicitly estimating uncertainty, or training separate exploration and exploitation policies. We revisit the dual-policy setting and ask whether optimistic exploration can be obtained without explicit uncertainty heads, learned optimism coefficients, or auxiliary exploration rewards. In this work, we propose BOXD, a simple dual-policy actor-critic method that trains an optimistic exploration policy alongside a conservative task policy. BOXD uses dropout-enabled critic ensembles and trains the exploration policy against the maximum over stochastic critic estimates. Under a local Gaussian approximation, this maximum behaves approximately like a UCB mean-plus-uncertainty objective, providing an optimistic training signal without explicitly estimating uncertainty. The task policy, meanwhile, remains trained with the conservative value targets used by the underlying actor-critic algorithm. During data collection, BOXD stochastically switches between the exploration and task policies, gradually increasing the use of the task policy as learning progresses. The shared replay buffer therefore accumulates both exploratory and task-aligned transitions, allowing each policy to benefit from the data collected by the other. Experiments on DeepMind Control Suite show that BOXD substantially improves SAC- and TD3-based agents, allowing them to remain competitive with heavily engineered state-of-the-art methods while retaining a relatively simple algorithmic structure. We find that BOXD leads to performance improvements even when combined with a strong XQC algorithm, along with gains on several tasks, and across these benchmarks it also compares favourably with related optimistic and dual-policy exploration baselines on several challenging tasks. These results suggest that dual-policy optimistic exploration can strengthen modern actor-critic algorithms and that exploration mechanisms remain a promising route for advancing state-of-the-art continuous-control performance.
URL: https://openreview.net/forum?id=eCFiOEi59l
---
Title: ReasonGen-R1: Chain-of-Thought Reasoning for Autoregressive Image Generation
Abstract: Although chain-of-thought (CoT) reasoning and reinforcement learning (RL) have driven major advances in large language models (LLMs), their integration into generative vision models remains largely unexplored. We introduce ReasonGen-R1, a two-stage framework that equips an autoregressive image generator with explicit text-based reasoning. To our knowledge, ReasonGen-R1 is the first autoregressive text-to-image model in which textual reasoning and image tokens are emitted by the same policy in a single autoregressive pass and optimized end-to-end. We first perform supervised fine-tuning (SFT) on a reasoning-augmented corpus of model-generated rationales paired with input prompts, allowing the model to plan object layouts, styles, and scene compositions prior to generation. We then apply reinforcement learning with Group Relative Policy Optimization (GRPO), where a single pretrained vision-language model provides a simple instruction-alignment reward on the final image only, removing the need for the expert ensembles or per-step process rewards used by prior reasoning-based generators. Because text and image tokens share one policy, GRPO can credit-assign through the CoT, turning it into an RL-optimizable interface between prompt understanding and visual synthesis. We further introduce an adaptive entropy loss that stabilizes training in this challenging interleaved multimodal setting. Across three instruction-following benchmarks (GenEval, DPG-Bench, and T2I-Benchmark), ReasonGen-R1 improves consistently over its autoregressive base model and performs on par with prior reasoning-based generators.
URL: https://openreview.net/forum?id=YnZJXGd1S7
---
Title: OCEAN: Observation Consensus over Evidence with Dynamic Routing for Retrieval-augmented Generation
Abstract: Retrieval-Augmented Generation (RAG) has substantially enhanced the reasoning capabilities of Large Language Models (LLMs) by incorporating external evidence retrieved from knowledge sources. However, leveraging multiple pieces of retrieved evidence inevitably incurs additional computational overhead, which has motivated growing interest in evidence compression for RAG. Existing methods typically compress evidence by either extracting salient contexts or generating evidence summaries. Nevertheless, such compression is inherently lossy: it discards fine-grained evidence and, in the abstractive case, may introduce hallucinations, while prior work largely overlooks how the discarded information can be recovered at decoding time. To address these limitations, we propose Observation Consensus over Evidence with dynAmic routiNg (OCEAN) for RAG, which compensates for the information loss of compression-based RAG at the decoding stage rather than by enlarging the input. The core idea of OCEAN is to combine a question-conditioned evidence summary with complementary multi-view evidence through uncertainty-aware dynamic logit routing. Specifically, alongside the compressed summary, OCEAN exploits a small high-fidelity subset of semantically similar evidence and the model's parametric knowledge as complementary information sources, and integrates these multi-view distributions via uncertainty-aware dynamic logit routing to produce the final response. Additionally, extensive experiments on four open-domain QA benchmarks with two diverse backbone models demonstrate the effectiveness and robustness of OCEAN.
URL: https://openreview.net/forum?id=osqbe0jyHZ
---
Title: Epipolar Geometry Improves Video Generation Models
Abstract: Video generation models have advanced significantly through the latent diffusion transformers trained with rectified flow techniques. Yet these models still struggle with geometric inconsistencies, unstable motion, and visual artifacts that break the illusion of realistic 3D scenes. 3D-consistent video generation could significantly impact numerous downstream applications in generation and reconstruction tasks. We explore how epipolar geometry constraints improve modern video diffusion models. Despite using massive training data, these models fail to capture fundamental geometric principles. We align diffusion models using pairwise epipolar geometry constraints via preference-based optimization, directly addressing unstable trajectories and geometric artifacts through mathematically principled geometric enforcement. Our approach efficiently enforces geometric principles without requiring end-to-end differentiability. Evaluation demonstrates that classical geometric constraints provide more stable optimization signals than modern learned metrics. Training on static scenes with dynamic cameras ensures metric quality while the model generalizes to various dynamic scenes. By bridging data-driven learning with classical computer vision, we reduce epipolar error by 31% and improve human-rated consistency from 54% to 72% without compromising visual quality.
URL: https://openreview.net/forum?id=xKvQakeoBP
---
Title: MAPGD: Multi-Agent Prompt Gradient Descent for Collaborative Prompt Optimization
Abstract: Prompt engineering is crucial for fully leveraging large language models, yet existing prompt optimization methods often rely on a single refinement trajectory, leading to instability, conflicting update signals, and inefficient use of query budgets. We propose Multi-Agent Prompt Gradient Descent (MAPGD), a gradient-inspired framework that coordinates multiple complementary prompt editing signals for robust and interpretable optimization. MAPGD decomposes prompt refinement into orthogonal dimensions (e.g., instruction clarity, example selection, format, and style) and aggregates textual pseudo-gradients via semantic embedding, conflict-aware clustering, and adaptive fusion. To improve robustness, we introduce HCGC, which enforces angular separation between conflicting directions, and CAAW, which dynamically calibrates editing contributions based on validation feedback. Experiments on diverse classification and reasoning benchmarks show that MAPGD achieves consistent gains in accuracy over strong baselines, while ablation results indicate that coordinated gradient fusion and adaptive weighting are critical to stable optimization. Together, these findings suggest that MAPGD offers a robust and interpretable framework for automatic prompt optimization.
URL: https://openreview.net/forum?id=3kmaMv2r2Z
---
Title: Inferring Network Controllability from Coarse Measurements
Abstract: The control of large-scale network dynamical systems is important in diverse settings, including data networks, traffic networks, and biological networks. Often our knowledge of network structure is not detailed at the finest scale of the dynamics, due to our inability to observe and characterize at such scales. Rather, our knowledge may be limited to coarse summary information. Our objective herein is to characterize the average controllability of a fine-scale linear time-invariant (LTI) system based on knowledge of only a coarsened approximation of that system. We provide conditions under which the average controllability can be well approximated by that of the (synthesized, reduced-order) coarse-scale system. Not surprisingly, our results show that for the approximation to be good, some inherent fine-scale parametric structure is required as well as a type of coherence –between the coarse observations and underlying structure. The parametric model we assume for the fine-scale network is the stochastic block model (SBM), developed for community detection.
We study two complementary methods: a classical structure-agnostic model-order reduction method (C-SaMOR) that treats the coarse graph as a surrogate dynamical system, and a learning-based SaMOR (Lb-SaMOR) that infers the latent community representation before computing controllability. We highlight regimes in which each can outperform the other. Finally, we provide simulations to confirm our theoretical results for different network sizes and densities, characterizing how much community structure is retained in the coarse
summary. Our work reveals conditions under which coarse measurements support accurate influence estimation for control and intervention in community-structured networked systems.
URL: https://openreview.net/forum?id=DBR1PkaiKw
---
Title: On Arbitrary Predictions from Equally Valid Models
Abstract: Model multiplicity describes the existence of multiple models that fit the data equally well but can produce different predictions on individual samples, so-called predictive multiplicity. In medicine, these models can admit conflicting predictions for the same patient—a risk that is poorly understood and insufficiently addressed. In this study, we empirically analyze predictive multiplicity across multiple medical tasks and model architectures and show practical strategies to mitigate it. Our analysis reveals that (1) standard validation metrics fail to identify a uniquely optimal model. (2) Models with equivalent population-level performance yield conflicting patient-level predictions, resulting in arbitrary and potentially harmful outcomes under any single model. (3) For already overparameterized models, higher capacity reduces predictive multiplicity when it improves accuracy. Disagreement between models is not at random and (4) can be leveraged as an indicator of genuine uncertainty, identifying patients for whom the current evidence is insufficient. Together, our findings highlight that predictive multiplicity is not a theoretical curiosity but a pervasive and practically significant issue in medical AI. We argue that accounting for multiplicity should be
considered a core component of model evaluation and deployment in safety-critical domains.
URL: https://openreview.net/forum?id=UX2bTXKGAd
---
Title: CoSeLECT: Adaptive Frame Selection for Video-Language Understanding
Abstract: Multimodal Large Language Models (MLLMs) have shown strong performance on image understanding tasks, but video comprehension remains a significant challenge due to the high computational cost of processing long frame sequences and the limited token capacity of underlying Large Language Models (LLMs). Prior approaches to address this often rely on uniform frame sampling, query-agnostic pruning, or require costly training of dedicated compression modules. In this work, we introduce CoSeLECT, a training-free, plug-and-play, query-guided frame selection method that intelligently subsamples video frames for efficient use in MLLMs. CoSeLECT leverages two key signals: temporal redundancy, which identifies similar frame clusters, and query relevance, which selects frames based on their semantic alignment with the input query. By combining these signals through an adaptive frame selection strategy, CoSeLECT selects frames that are both diverse and highly relevant to the query, without requiring any model-specific tuning. Our results on various base MLLMs show that CoSeLECT consistently outperforms trained and training-free state-of-the-art methods, including LongVU by +3.8% on MLVU and AKS by +4.5% on EgoSchema.
URL: https://openreview.net/forum?id=Wd3QdEAlje
---