Weekly TMLR digest for Mar 29, 2026
TMLR
New certifications
==================
J2C Certification: Transformers in the Dark: Navigating Unknown Search Spaces via Bandit Feedback
Jungtaek Kim, Thomas Zeng, Ziqian Lin, Minjae Lee, Chungpa Lee, Jy-yong Sohn, Hyung Il Koo, Kangwook Lee
https://openreview.net/forum?id=Jij7zCjVfc
---
Expert Certification: FedLOE: Federated Domain Generalization via Locally Overfit Ensemble
Ruqi Bai, David I. Inouye
https://openreview.net/forum?id=W4T9sK6Gai
---
Survey Certification: Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challenges
Kemal Oksuz, Alexandru Buburuzan, Anthony Knittel, Yuhan Yao, Puneet K. Dokania
https://openreview.net/forum?id=E2L5J2O2Bk
---
J2C Certification: $\texttt{SEM-CTRL}$: Semantically Controlled Decoding
Mohammad Albinhassan, Pranava Madhyastha, Alessandra Russo
https://openreview.net/forum?id=ICUHKhOISN
---
Accepted papers
===============
Title: Learning Representations for Independence Testing
Authors: Nathaniel Xu, Feng Liu, Danica J. Sutherland
Abstract: Many tools exist to detect dependence between random variables, a core question across a wide range of machine learning, statistical, and scientific endeavors. Although several statistical tests guarantee eventual detection of any dependence with enough samples, standard tests may require an exorbitant amount of samples for detecting subtle dependencies between high-dimensional random variables with complex distributions. In this work, we study two related ways to learn powerful independence tests. First, we show how to construct powerful statistical tests with finite-sample validity by using variational estimators of mutual information, such as the InfoNCE or NWJ estimators. Second, we establish a close connection between these variational mutual information-based tests and tests based on the Hilbert-Schmidt Independence Criterion (HSIC); in particular, learning a variational bound (typically parameterized by a deep network) for mutual information is closely related to learning a kernel for HSIC. Finally, we show how to, rather than selecting a representation to maximize the statistic itself, select a representation which can maximize the power of a test, in either setting; we term the former case a Neural Dependency Statistic (NDS). While HSIC power optimization has been recently considered in the literature, we correct some important misconceptions and expand to considering deep kernels. In our experiments, while all approaches can yield powerful tests with exact level control, optimized HSIC tests generally outperform the other approaches on difficult problems of detecting structured dependence.
URL: https://openreview.net/forum?id=pDvKoXRsnW
---
Title: Uncertainty-Aware Systems for Human-AI Collaboration
Authors: Vasco Pearson, Jean V. Alves, Jacopo Bono, Mario A. T. Figueiredo, Pedro Bizarro
Abstract: \textit{Learning to defer} (\textbf{L2D}) algorithms improve human-AI collaboration (\textbf{HAIC}) by deferring decisions to human experts when they are more likely to be correct than the AI model. This framework hinges on machine learning (\textbf{ML}) models' ability to assess their own certainty and that of human experts. L2D struggles in dynamic environments, where distribution shifts impair deferral.
We argue that robust HAIC in dynamic environments requires uncertainty-driven policy switching rather than reliance on a single deferral strategy. To operationalize this principle, we introduce two uncertainty-aware approaches that estimate epistemic uncertainty to guide the deferral policy choice. Both methods are the first uncertainty-aware approaches for HAIC that also address limitations of L2D systems including cost-sensitive scenarios, limited human predictions, and capacity constraints. Empirical evaluation in fraud detection shows both approaches outperform state-of-the-art baselines while improving calibration and supporting real-world adoption.
URL: https://openreview.net/forum?id=PiRYCyNBqQ
---
Title: Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey
Authors: Bhavuk Jain, Sercan O Arik, HARDEO KUMAR THAKUR
Abstract: Multimodal large language models (MLLMs) integrate information from multiple modalities such as text, images, audio, and video, enabling complex capabilities such as visual question answering and audio translation. While powerful, this increased expressiveness introduces new and amplified vulnerabilities to adversarial manipulation. This survey provides a comprehensive and systematic analysis of adversarial threats to MLLMs, moving beyond enumerating attack techniques to explain the underlying causes of model susceptibility. We introduce a taxonomy that organizes adversarial attacks according to attacker objectives, unifying diverse attack surfaces across modalities and deployment settings. Additionally, we also present a vulnerability-centric analysis that links integrity attacks, safety and jailbreak
failures, control and instruction hijacking, and training-time poisoning to shared architectural and representational weaknesses in multimodal systems. Together, this framework provides an explanatory foundation for understanding adversarial behavior in MLLMs and
informs the development of more robust and secure multimodal language systems.
URL: https://openreview.net/forum?id=zwzodDJkzZ
---
Title: Distilled Circuits: A Mechanistic Study of Internal Restructuring in Knowledge Distillation
Authors: Reilly Haskins, Benjamin Adams
Abstract: Knowledge distillation compresses a larger neural model (teacher) into smaller, faster student models by training them to match teacher outputs. However, the internal computational transformations that occur during this process remain poorly understood. We apply techniques from mechanistic interpretability to analyze how internal circuits, representations, and activation patterns differ between teachers and students. Focusing on GPT2 and its distilled counterpart DistilGPT2, and generalizing our findings to both bidirectional architectures and larger model pairs, we find that student models can reorganize, compress, and discard teacher components, often resulting in a stronger reliance on fewer individual components. To quantify functional alignment beyond output similarity, we introduce an alignment metric based on influence-weighted component similarity, validated across multiple tasks. Our findings reveal that while knowledge distillation preserves broad functional behaviors, it also causes significant shifts in internal computation, with important implications for the robustness and generalization capacity of distilled models.
URL: https://openreview.net/forum?id=S1KJE2ZW64
---
Title: FIT-GNN: Faster Inference Time for GNNs that ‘FIT’ in Memory Using Coarsening
Authors: Shubhajit Roy, Hrriday Ruparel, Kishan Ved, Anirban Dasgupta
Abstract: Scalability of Graph Neural Networks (GNNs) remains a significant challenge. To tackle this, methods like coarsening, condensation, and computation trees are used to train on a smaller graph, resulting in faster computation. Nonetheless, prior research has not adequately addressed the computational costs during the inference phase. This paper presents a novel approach to improve the scalability of GNNs by reducing computational burden during the inference phase using graph coarsening. We demonstrate two different methods -- Extra Nodes and Cluster Nodes. Our study extends the application of graph coarsening for graph-level tasks, including graph classification and graph regression. We conduct extensive experiments on multiple benchmark datasets to evaluate the performance of our approach. Our results show that the proposed method achieves orders of magnitude improvements in single-node inference time compared to traditional approaches. Furthermore, it significantly reduces memory consumption for node and graph classification and regression tasks, enabling efficient training and inference on low-resource devices where conventional methods are impractical. Notably, these computational advantages are achieved while maintaining competitive performance relative to baseline models.
URL: https://openreview.net/forum?id=g7r7y2I7Sz
---
Title: One Model for All: Multi-Objective Controllable Language Models
Authors: Qiang He, Yucheng Yang, Tianyi Zhou, Meng Fang, Mykola Pechenizkiy, Setareh Maghsudi
Abstract: Aligning large language models (LLMs) with human preferences is critical for enhancing LLMs' safety, helpfulness, humor, faithfulness, etc. Current reinforcement learning from human feedback (RLHF) mainly focuses on a fixed reward learned from average human ratings, which may weaken the adaptability and controllability of varying preferences. However, creating personalized LLMs requires aligning LLMs with individual human preferences, which is non-trivial due to the scarce data per user and the diversity of user preferences in multi-objective trade-offs, varying from emphasizing empathy in certain contexts to demanding efficiency and precision in others. Can we train one LLM to produce personalized outputs across different user preferences on the Pareto front? In this paper, we introduce Multi-Objective Control (MOC), which trains a single LLM to directly generate responses in the preference-defined regions of the Pareto front. Our approach introduces multi-objective optimization (MOO) principles into RLHF to train an LLM as a preference-conditioned policy network. We improve the computational efficiency of MOC by applying MOO at the policy level, enabling us to fine-tune a 7B-parameter model on a single A6000 GPU. Extensive experiments demonstrate the advantages of MOC over baselines in three aspects: (i) controllability of LLM outputs w.r.t. user preferences on the trade-off among multiple rewards; (ii) quality and diversity of LLM outputs, measured by the hyper-volume of multiple solutions achieved; and (iii) generalization to unseen preferences. These results highlight MOC's potential for real-world applications requiring scalable and customizable LLMs.
URL: https://openreview.net/forum?id=qAM5PmvFYY
---
Title: One Pic is All it Takes: Poisoning Visual Document Retrieval Augmented Generation with a Single Image
Authors: Ezzeldin Shereen, Dan Ristea, Shae McFadden, Burak Hasircioglu, Vasilios Mavroudis, Chris Hicks
Abstract: Retrieval-augmented generation (RAG) is instrumental for inhibiting hallucinations in large language models (LLMs) through the use of a factual knowledge base (KB). Although PDF documents are prominent sources of knowledge, text-based RAG pipelines are ineffective at capturing their rich multi-modal information. In contrast, visual document RAG~(VD-RAG) uses screenshots of document pages as the KB, which has been shown to achieve state-of-the-art results. However, by introducing the image modality, VD-RAG introduces new attack vectors for adversaries to disrupt the system by injecting malicious documents into the KB. In this paper, we demonstrate the vulnerability of VD-RAG to poisoning attacks targeting both retrieval and generation. We define two attack objectives and demonstrate that both can be realized by injecting only a single adversarial image into the KB. Firstly, we introduce a targeted attack against one or a group of queries with the goal of spreading targeted disinformation. Secondly, we present a universal attack that, for any potential user query, influences the response to cause a denial-of-service in the VD-RAG system. We investigate the two attack objectives under both white-box and black-box assumptions, employing a multi-objective gradient-based optimization approach as well as prompting state-of-the-art generative models. Using two visual document datasets, a diverse set of state-of-the-art retrievers~(embedding models) and generators~(vision language models), we show VD-RAG is vulnerable to poisoning attacks in both the targeted and universal settings, yet demonstrating robustness to black-box attacks in the universal setting.
URL: https://openreview.net/forum?id=CLkjUidlYg
---
Title: Transformers in the Dark: Navigating Unknown Search Spaces via Bandit Feedback
Authors: Jungtaek Kim, Thomas Zeng, Ziqian Lin, Minjae Lee, Chungpa Lee, Jy-yong Sohn, Hyung Il Koo, Kangwook Lee
Abstract: Effective problem solving with Large Language Models (LLMs) can be enhanced when they are paired with external search algorithms. By viewing the space of diverse ideas and their follow-up possibilities as a tree structure, the search algorithm can navigate such a search space and guide the LLM toward better solutions more efficiently. While the search algorithm enables an effective balance between exploitation and exploration of a tree-structured space, the need for an external component can complicate the overall problem-solving process. We therefore pose the following question: Can LLMs or their underlying Transformer architectures approximate a search algorithm? To answer this question, we first introduce a simplified framework in which tree extensions and feedback signals are externally specified, allowing for controlled evaluation of search capabilities. We call this setting unknown tree search with bandit feedback. Within this setting, we show that Transformers are theoretically expressive enough to implement distinct search strategies and can be trained from scratch to approximate those strategies. Our Transformer models exhibit the possibility of generalizing to unseen conditions such as longer horizons or deeper trees. Furthermore, we demonstrate that continued task-focused training unlocks the complete capabilities of a pretrained LLM, by fine-tuning the LLM on search trajectories.
URL: https://openreview.net/forum?id=Jij7zCjVfc
---
Title: A Bayesian Approach to Segmentation with Noisy Labels via Spatially Correlated Distributions
Authors: Ryu Tadokoro, Tsukasa Takagi, Shin-ichi Maeda
Abstract: In semantic segmentation, the accuracy of models heavily depends on the high-quality annotations. However, in many practical scenarios, such as medical imaging and remote sensing, obtaining true annotations is not straightforward and usually requires significant human labor.
Relying on human labor often introduces annotation errors, including mislabeling, omissions, and inconsistency between annotators.
In the case of remote sensing, differences in procurement time can lead to misaligned ground-truth annotations.
These label errors are not independently distributed, and instead usually appear in spatially connected regions where adjacent pixels are more likely to share the same errors.
To address these issues, we propose an approximate Bayesian estimation based on a probabilistic model that assumes training data include label errors, incorporating the tendency for these errors to occur with spatial correlations between adjacent pixels.
However, Bayesian inference for such spatially correlated discrete variables is notoriously intractable. To overcome this fundamental challenge, we introduce a novel class of probabilistic models, which we term the \textbf{ELBO-Computable Correlated Discrete Distribution (ECCD)}. By representing the discrete dependencies through a continuous latent Gaussian field with a Kac-Murdock-Szeg\"{o} (KMS) structured covariance, our framework enables scalable and efficient variational inference for problems previously considered computationally prohibitive.
Through experiments on multiple segmentation tasks, we confirm that leveraging the spatial correlation of label errors improves robustness.
Notably, in specific tasks such as lung segmentation, the proposed method achieves performance comparable to training with clean labels under moderate noise levels.
URL: https://openreview.net/forum?id=oMgfr8Kk2x
---
Title: Cost-Aware Routing for Efficient Text-To-Image Generation
Authors: Qinchan Li, Kenneth Chen, Changyue Su, Wittawat Jitkrittum, Qi Sun, Patsorn Sangkloy
Abstract: Diffusion models are well known for their ability to generate a high-fidelity image for an input
prompt through an iterative denoising process. Unfortunately, the high fidelity also comes at
a high computational cost due to the inherently sequential generative process. In this work,
we seek to optimally balance quality and computational cost, and propose a framework to
allow the amount of computation to vary for each prompt, depending on its complexity. Each
prompt is automatically routed to the most appropriate text-to-image generation function,
which may correspond to a distinct number of denoising steps of a diffusion model, or a
disparate, independent text-to-image model. Unlike uniform cost reduction techniques (e.g.,
distillation, model quantization), our approach achieves the optimal trade-off by learning to
reserve expensive choices (e.g., 100+ denoising steps) only for a few complex prompts, and
employ more economical choices (e.g., small distilled model) for less sophisticated prompts.
We empirically demonstrate on COCO and DiffusionDB that by learning to route to nine
already-trained text-to-image models, our approach is able to deliver an average quality
that is higher than that achievable by any of these models alone.
URL: https://openreview.net/forum?id=Jbe9AVsYS6
---
Title: FedLOE: Federated Domain Generalization via Locally Overfit Ensemble
Authors: Ruqi Bai, David I. Inouye
Abstract: In federated learning (FL), clients typically access data from just one distribution. Ideally, the learned models would generalize to out-of-distribution (OOD) data, i.e., domain generalization (DG). However, centralized DG methods cannot easily be adapted to the domain separation context, and some existing federated DG methods can be brittle in the large-client, domain-separated regime. To address these challenges, we revisit the classic mixture-of-experts (MoE) idea by viewing each client as an expert on its own dataset. From this perspective, simple federated averaging can be seen as a type of iterative MoE, where the amount of local training determines the strength of each expert. In contrast to the standard FL communication-performance trade-off, we theoretically demonstrate in linear cases and empirically validate in deep models that reducing communication frequency can effectively enhance DG performance, surpassing centralized counterparts (e.g., $+4.34\%$ on PACS). Building on this, we further propose an additional MoE strategy to combine the client-specific classifier heads using standard DG objectives. Our proposed \methodname method can be viewed as an intermediate approach between FedAvg and one-time ensembling. It demonstrates both theoretical soundness and empirical effectiveness. Moreover, \methodname requires fewer communication rounds, highlighting its practical efficiency and scalability.
URL: https://openreview.net/forum?id=W4T9sK6Gai
---
Title: SpikingBrain: Spiking Brain-inspired Large Models
Authors: Yuqi Pan, Yupeng Feng, JingHao Zhuang, siyu ding, Han Xu, Zehao Liu, Bohan Sun, Yuhong Chou, Xuerui Qiu, Anlin Deng, Anjie Hu, Shurong Wang, Peng Zhou, Man Yao, Jibin Wu, jian yang, 孙国梁, Bo XU, Guoqi Li
Abstract: Mainstream Transformer-based large language models (LLMs) face significant efficiency bottlenecks: training computation scales quadratically with sequence length, and inference memory grows linearly. These constraints limit their ability to process long sequences effectively. In addition, building large models on non-NVIDIA computing platforms poses major challenges in achieving stable and efficient training and deployment. To address these issues, we introduce SpikingBrain, a new family of brain-inspired models designed for efficient long-context training and inference. SpikingBrain leverages the MetaX GPU cluster and focuses on three core aspects: (1) Model Architecture: linear and hybrid-linear attention architectures with adaptive spiking neurons; (2) Algorithmic Optimizations: an efficient, conversion-based training pipeline compatible with existing LLMs, along with a dedicated spike coding framework; (3) System Engineering: customized training frameworks, operator libraries, and parallelism strategies tailored to the MetaX hardware. Using these techniques, we develop two models: SpikingBrain-7B, a linear LLM, and SpikingBrain-76B, a hybrid-linear MoE LLM. These models demonstrate the feasibility of large-scale LLM development on non-NVIDIA platforms, and our training framework supports weeks of stable training on hundreds of MetaX GPUs with Model FLOPs Utilization (MFU) at expected levels. SpikingBrain achieves performance comparable to open-source Transformer baselines while using exceptionally low data resources (continual pre-training of approximately 150B tokens). Our models also significantly improve long-context efficiency and deliver inference with (partially) constant memory and event-driven spiking behavior. For example, SpikingBrain-7B achieves more than 100× speedup in Time to First Token (TTFT) for 4M-token sequences. Furthermore, the proposed spiking scheme achieves 69.15% sparsity, enabling low-power operation. Overall, this work demonstrates the potential of brain-inspired mechanisms to drive the next generation of efficient and scalable large model design.
URL: https://openreview.net/forum?id=PNLShc0C6q
---
Title: PrismBench: Dynamic and Flexible Benchmarking of LLMs Code Generation with Monte Carlo Tree Search
Authors: Vahid Majdinasab, Amin Nikanjam, Foutse Khomh
Abstract: The rapid advancement of LLMs' code generation capabilities is outpacing traditional evaluation methods. Static benchmarks fail to capture the depth and breadth of LLM capabilities and eventually become obsolete, while most dynamic approaches either rely too heavily on LLM-based evaluation or remain constrained by predefined test sets. To address these issues, we introduce PrismBench, a multi-agent, dynamic benchmarking framework designed to systematically expose and analyze LLM failure modes in code generation tasks. We formulate evaluation as a Markov Decision Process over a structured tree of coding challenges, leveraging a customized Monte Carlo Tree Search algorithm to traverse this tree and discover high-failure scenarios. Our multi-agent setup orchestrates task generation, model response, and analysis, enabling scalable assessment across diverse coding challenges. Additionally, we propose metrics that combine structural traversal patterns with performance across different tasks and difficulty levels to enable diagnostic and systematic comparison of LLMs' performance. We conduct extensive experiments on eight state-of-the-art LLMs and analyze how model architecture and scale influence code generation performance across varying coding tasks. All code, evaluation trees, and a public leaderboard are available at https://prismbench.github.io/Demo/
URL: https://openreview.net/forum?id=O0bsC6FDly
---
Title: Graph Concept Bottleneck Models
Authors: Haotian Xu, Tsui-Wei Weng, Lam M. Nguyen, Tengfei Ma
Abstract: Concept Bottleneck Models (CBMs) have emerged as a prominent framework for interpretable deep learning, providing human-understandable intermediate concepts that enable transparent reasoning and direct intervention. However, existing CBMs typically assume conditional independence among concepts given the label, overlooking the intrinsic dependencies and correlations that often exist among them. In practice, concepts are rarely isolated: modifying one concept may inherently influence others. Ignoring these relationships can lead to oversimplified representations and weaken interpretability. To address this limitation, we introduce **Graph CBMs**, a novel variant of CBMs that explicitly models the relational structure among concepts through a latent concept graph. Our approach can be seamlessly integrated into existing CBMs as a lightweight, plug-and-play module, enriching their reasoning capability without sacrificing interpretability. Experimental results on multiple real-world image classification benchmarks demonstrate that Graph CBMs (1) achieve higher predictive accuracy while revealing meaningful concept structures, (2) enable more effective and robust concept-level interventions, and (3) maintain stable performance across diverse architectures and training setups.
URL: https://openreview.net/forum?id=a4azUYjRhU
---
Title: Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challenges
Authors: Kemal Oksuz, Alexandru Buburuzan, Anthony Knittel, Yuhan Yao, Puneet K. Dokania
Abstract: The emergence of multi-modal foundation models has markedly transformed the technology for autonomous driving, shifting away from conventional and mostly hand-crafted design choices towards unified, foundation-model-based approaches, capable of directly inferring motion trajectories from raw sensory inputs. This new class of methods can also incorporate natural language as an additional modality, with Vision-Language-Action (VLA) models serving as a representative example. In this review, we provide a comprehensive examination of such methods through a unifying taxonomy to critically evaluate their architectural design choices, methodological strengths, and their inherent capabilities and limitations. Our survey covers 37 recently proposed approaches that span the landscape of trajectory planning with foundation models. Furthermore, we assess these approaches with respect to the openness of their source code and datasets, offering valuable information to practitioners and researchers. We provide an accompanying webpage that catalogues the methods based on our taxonomy, available at: https://github.com/fiveai/FMs-for-driving-trajectories
URL: https://openreview.net/forum?id=E2L5J2O2Bk
---
Title: Learning to Localize Leakage of Cryptographic Sensitive Variables
Authors: Jimmy Gammell, Anand Raghunathan, Abolfazl Hashemi, Kaushik Roy
Abstract: While cryptographic algorithms such as the ubiquitous Advanced Encryption Standard (AES) are secure, *physical implementations* of these algorithms in hardware inevitably `leak' sensitive data such as cryptographic keys. A particularly insidious form of leakage arises from the fact that hardware consumes power and emits radiation in a manner that is statistically associated with the data it processes and the instructions it executes. Supervised deep learning has emerged as a state-of-the-art tool for carrying out *side-channel attacks*, which exploit this leakage by learning to map power/radiation measurements throughout encryption to the sensitive data operated on during that encryption. In this work we develop a principled deep learning framework for determining the relative leakage due to measurements recorded at different points in time, in order to inform *defense* against such attacks. This information is invaluable to cryptographic hardware designers for understanding *why* their hardware leaks and how they can mitigate it (e.g. by indicating the particular sections of code or electronic components which are responsible). Our framework is based on an adversarial game between a classifier trained to estimate the conditional distributions of sensitive data given subsets of measurements, and a budget-constrained noise distribution which probabilistically erases individual measurements to maximize the loss of this classifier. We demonstrate our method’s efficacy and ability to overcome limitations of prior work through extensive experimental comparison on 6 publicly-available power/EM trace datasets from AES, ECC and RSA implementations. Our PyTorch code is available at https://github.com/jimgammell/learning_to_localize_leakage.
URL: https://openreview.net/forum?id=9qxCSU8nDO
---
Title: Reward Engineering for Spatial Epidemic Simulations: A Reinforcement Learning Platform for Individual Behavioral Learning
Authors: Radman Rakhshandehroo, Daniel Coombs
Abstract: We present ContagionRL, a Gymnasium-compatible reinforcement learning platform specifically designed for systematic reward engineering in spatial epidemic simulations. Unlike traditional agent-based models that rely on fixed behavioral rules, our platform enables rigorous evaluation of how reward function design affects learned survival strategies across diverse epidemic scenarios. ContagionRL integrates a spatial SIRS+D epidemiological model with configurable environmental parameters, allowing researchers to stress-test reward functions under varying conditions including limited observability, different movement patterns, and heterogeneous population dynamics. We evaluate five distinct reward designs, ranging from sparse survival bonuses to a novel potential field approach, across multiple RL algorithms (PPO, SAC, A2C). Through systematic ablation studies, we identify that directional guidance and explicit adherence incentives are critical components for robust policy learning. Our comprehensive evaluation across varying infection rates, grid sizes, visibility constraints, and movement patterns reveals that reward function choice dramatically impacts agent behavior and survival outcomes. Agents trained with our potential field reward consistently achieve superior performance, learning maximal adherence to non-pharmaceutical interventions while developing sophisticated spatial avoidance strategies. The platform's modular design enables systematic exploration of reward-behavior relationships, addressing a knowledge gap in models of this type where reward engineering has received limited attention. ContagionRL is an effective platform for studying adaptive behavioral responses in epidemic contexts and highlight the importance of reward design, information structure, and environmental predictability in learning. Our code is publicly available at https://github.com/redradman/ContagionRL
URL: https://openreview.net/forum?id=yPEASsx3hk
---
Title: $\texttt{SEM-CTRL}$: Semantically Controlled Decoding
Authors: Mohammad Albinhassan, Pranava Madhyastha, Alessandra Russo
Abstract: Ensuring both syntactic and semantic correctness in Large Language Model (LLM) outputs remains a significant challenge, despite being critical for real-world deployment. In this paper, we introduce $\texttt{SEM-CTRL}$, a unified approach that allows for enforcing rich context-sensitive constraints, and task and instance specific semantics directly on the LLM decoder. Our approach integrates token-level MCTS which is guided by specific syntactic and semantic constraints. The constraints over desired outputs are expressed using Answer Set Grammars, which is a logic-based formalism that generalizes context sensitive grammars while incorporating background knowledge to represent task-specific semantics. We show that our approach helps guarantee valid completions for any off-the-shelf LLM without the need for fine-tuning. We evaluate $\texttt{SEM-CTRL}$ on a range of tasks, including synthetic grammar synthesis, combinatorial reasoning, JSON parsing, and planning. Our experimental results demonstrate that $\texttt{SEM-CTRL}$ allows even small pre-trained LLMs to efficiently outperform larger variants and state-of-the-art reasoning models (e.g., $\text{\textit{o4-mini}}$) while simultaneously guaranteeing semantic validity.
URL: https://openreview.net/forum?id=ICUHKhOISN
---
Title: Prioritizing Image-Related Tokens Enhances Vision-Language Pre-Training
Authors: Yangyi Chen, Hao Peng, Tong Zhang, Heng Ji
Abstract: In standard large vision-language models (LVLMs) pre-training, the model typically maximizes the joint probability of the caption conditioned on the image via next-token prediction (NTP); however, since only a small subset of caption tokens directly relates to the visual content, this naive NTP unintentionally fits the model to noise and increases the risk of hallucination. We present PRIOR, a simple vision-language pre-training approach that addresses this issue by prioritizing image-related tokens through differential weighting in the NTP loss, drawing from the importance sampling framework. PRIOR introduces a reference model—a text-only large language model (LLM) trained on the captions without image inputs, to weight each token based on its probability for LVLMs training. Intuitively, tokens that are directly related to the visual inputs are harder to predict without the image and thus receive lower probabilities from the text-only reference LLM. During training, we implement a token-specific re-weighting term based on the importance scores to adjust each token's loss. We implement PRIOR in two distinct settings: LVLMs with visual encoders and LVLMs without visual encoders. We observe 19% and 8% average relative improvement, respectively, on several vision-language benchmarks compared to NTP. In addition, PRIOR exhibits superior scaling properties, as demonstrated by significantly higher scaling coefficients, indicating greater potential for performance gains compared to NTP given increasing compute and data. The code is available at https://github.com/Yangyi-Chen/PRIOR.
URL: https://openreview.net/forum?id=jDcnL1hB1Z
---
Title: Automated Attention Pattern Discovery at Scale in Large Language Models
Authors: Jonathan Katzy, Razvan Mihai Popescu, Erik Mekkes, Arie van Deursen, Maliheh Izadi
Abstract: Large language models have found their success by scaling up their capabilities to work in general settings. The same can unfortunately not be said for their interpretability methods. The current trend in mechanistic interpretability is to provide precise explanations of specific behaviors in controlled settings. These often do not generalize well into other settings, or are too resource intensive for larger studies. In this work we propose to study repeated behaviors in large language models by mining completion scenarios in Java code datasets, through exploiting the structured nature of source code. We then collect the attention patterns generated in the attention heads to demonstrate that they are scalable signals for global interpretability of model components.
We show that vision models offer a promising direction for analyzing attention patterns at scale. To demonstrate this, we introduce the Attention Pattern – Masked Autoencoder (AP-MAE), a vision transformer-based model that efficiently reconstructs masked attention patterns. Experiments on StarCoder2 models (3B–15B) show that AP-MAE (i) reconstructs masked attention patterns with high accuracy, (ii) generalizes across unseen models with minimal degradation, (iii) reveals recurring patterns across a large number of inferences, (iv) predicts whether a generation will be correct without access to ground truth, with accuracies ranging from 55% to 70% depending on the task, and (v) enables targeted interventions that increase accuracy by 13.6% when applied selectively, but cause rapid collapse when applied excessively.
These results establish attention patterns as a scalable signal for interpretability and demonstrate that AP-MAE provides a transferable foundation for both analysis and intervention in large language models. Beyond its standalone value, AP-MAE can also serve as a selection procedure to guide more fine-grained mechanistic approaches toward the most relevant components. We release code and models to support future work in large-scale interpretability.
URL: https://openreview.net/forum?id=KpsUN0HAx7
---
Title: Covariance Density Neural Networks
Authors: Om Roy, Yashar Moshfeghi, Keith M Smith
Abstract: Graph neural networks have re-defined how we model and predict on network data but
there lacks a consensus on choosing the correct underlying graph structure on which to
model signals. CoVariance Neural Networks (VNN) address this issue by using the sample
covariance matrix as a Graph Shift Operator (GSO). Here, we improve on the performance
of VNNs by constructing a Density Matrix where we consider the sample Covariance matrix
as a quasi-Hamiltonian of the system in the space of random variables. Crucially, using this
density matrix as the GSO allows components of the data to be extracted at different scales,
allowing enhanced discriminability and performance. We show that this approach allows
explicit control of the stability-discriminability trade-off of the network, provides enhanced
robustness to noise compared to VNNs, and outperforms them in useful real-life applications
where the underlying covariance matrix is informative. In particular, we show that our
model can achieve strong performance in subject-independent Brain Computer Interface
EEG motor imagery classification, outperforming EEGnet while being faster. This shows
how covariance density neural networks provide a basis for the notoriously difficult task of
transferability of BCIs when evaluated on unseen individuals.
URL: https://openreview.net/forum?id=TwCkGi5XFB
---
Title: Offline Reinforcement Learning via Inverse Optimization
Authors: Ioannis Dimanidis, Tolga Ok, Peyman Mohajerin Esfahani
Abstract: Inspired by the recent successes of Inverse Optimization (IO) across various application domains, we propose a novel offline Reinforcement Learning (ORL) algorithm for continuous state and action spaces, leveraging the convex loss function called "sub-optimality loss" from the IO literature. To mitigate the distribution shift commonly observed in ORL problems, we further employ a robust and non-causal Model Predictive Control (MPC) expert steering a nominal model of the dynamics using in-hindsight information stemming from the model mismatch. Unlike the existing literature, our robust MPC expert enjoys an exact and tractable convex reformulation. In the second part of this study, we show that the IO hypothesis class, trained by the proposed convex loss function, enjoys ample expressiveness and reliably recovers teacher behavior in MuJoCo benchmarks. The method achieves competitive results compared to widely-used baselines in sample-constrained settings, despite using orders of magnitude fewer parameters. To facilitate the reproducibility of our results, we provide an open-source package implementing the proposed algorithms and the experiments. The code is available at https://github.com/TolgaOk/offlineRLviaIO.
URL: https://openreview.net/forum?id=lSj2gXoeCy
---
Title: Muon Optimizes Under Spectral Norm Constraints
Authors: Lizhang Chen, Jonathan Li, qiang liu
Abstract: The pursuit of faster optimization algorithms remains an active and important research direction in deep learning. Recently, the Muon optimizer has demonstrated promising empirical performance, but its theoretical foundation remains less understood. In this paper, we bridge this gap and provide a theoretical analysis of Muon by placing it within the Lion-$\mathcal{K}$ family of optimizers. Specifically, we show that Muon corresponds to Lion-$\mathcal{K}$ when equipped with the nuclear norm, and we leverage the theoretical results of Lion-$\mathcal{K}$ to establish that Muon (with decoupled weight decay) implicitly solves an optimization problem that enforces a constraint on the spectral norm of weight matrices. This perspective not only demystifies the implicit regularization effects of Muon but also leads to natural generalizations through varying the choice of convex map $\mathcal{K}$, allowing for the exploration of a broader class of implicitly regularized and constrained optimization algorithms.
URL: https://openreview.net/forum?id=Blz4hjxLwU
---
Title: ParaBlock: Communication-Computation Parallel Block Coordinate Federated Learning for Large Language Models
Authors: Yujia Wang, Yuanpu Cao, Jinghui Chen
Abstract: Federated learning (FL) has been extensively studied as a privacy-preserving training paradigm. Recently, federated block coordinate descent scheme has become a popular option in training large-scale models, as it allows clients to train only a subset of the model locally instead of the entire model. However, in the era of large language models (LLMs), even a single block can contain a significant number of parameters, posing substantial communication latency, particularly for resource-constrained clients. To address this challenge in federated training/fine-tuning LLMs, we propose ParaBlock, a novel approach that establishes two parallel threads for communication and computation to enhance communication efficiency. We theoretically prove that the proposed ParaBlock achieves the same convergence rate as the standard federated block coordinate descent methods. Empirical evaluations on fine-tuning LLMs on general instruction following and mathematical reasoning confirm that ParaBlock not only maintains strong performance but also significantly improves communication efficiency.
URL: https://openreview.net/forum?id=Hnf7eCdBeV
---
Title: Unlearning in Diffusion models under Data Constraints: A Variational Inference Approach
Authors: Subhodip Panda, Varun M S, Shreyans Jain, Sarthak Kumar Maharana, Prathosh AP
Abstract: For a responsible and safe deployment of diffusion models in various domains, regulating the generated outputs from these models is desirable because such models could generate undesired, violent, and obscene outputs. To tackle this problem, recent works use machine unlearning methodology to forget training data points containing these undesired features from pre-trained generative models. However, these methods proved to be ineffective in data-constrained settings where the whole training dataset is inaccessible. Thus, the principal objective of this work is to propose a machine unlearning methodology that can prevent the generation of outputs containing undesired features from a pre-trained diffusion model in such a data-constrained setting. Our proposed method, termed as Variational Diffusion Unlearning (**VDU**), is a computationally efficient method that only requires access to a subset of training data containing undesired features. Our approach is inspired by the variational inference framework with the objective of minimizing a loss function consisting of two terms: *plasticity inducer* and *stability regularizer*. *Plasticity inducer* reduces the log-likelihood of the undesired training data points, while the *stability regularizer*, essential for preventing loss of image generation quality, regularizes the model in parameter space. We validate the effectiveness of our method through comprehensive experiments for both class unlearning and feature unlearning. For class unlearning, we unlearn some user-identified classes from MNIST, CIFAR-10, and tinyImageNet datasets from a pre-trained unconditional denoising diffusion probabilistic model (DDPM). Similarly, for feature unlearning, we unlearn the generation of certain high-level features from a pre-trained Stable Diffusion model.
URL: https://openreview.net/forum?id=mAHRgieyOV
---
Title: Generalizing Coverage Plots for Simulation-based Inference
Authors: Maximilian Lipp, Benjamin Kurt Miller, Lyubov Amitonova, Patrick Forré
Abstract: Simulation-based inference (SBI) aims to find the probabilistic inverse of a non-linear function
by fitting the posterior with a generative model on samples. Applications demand accurate
uncertainty quantification, which can be difficult to achieve and verify. Since the ground
truth model is implicitly defined in SBI, we cannot compute likelihood values nor draw
samples from the posterior. This renders two-sample testing against the posterior impossible
for any practical use and calls for proxy verification methods such as expected coverage
testing. We introduce a differentiable objective that encourages coverage in the generative
model by parameterizing the dual form of the total variation norm with neural networks.
However, we find that coverage tests can easily report a good fit when the approximant
deviates significantly from the target distribution and give strong empirical evidence and
theoretical arguments why the expected coverage plot is, in general, not a reliable indicator
of posterior fit. To address this matter, we introduce a new ratio coverage plot as a better
alternative to coverage, which is not susceptible to the same blind spots. It comes at the
price of estimating a ratio between our model and the ground truth posterior, which can be
done using standard algorithms. We provide experimental results that back up this claim,
and provide multiple algorithms for estimating ratio coverage.
URL: https://openreview.net/forum?id=eqYapbG2jO
---
New submissions
===============
Title: Symmetry-Protected Geometric Separation for Verified Algebraic Reasoning
Abstract: Many reasoning problems are defined by algebraic invariance: multiple surface forms represent the same computation, and correctness must remain stable under semantics-preserving rewrites and increasing compositional depth---a regime where standard sequence models can exhibit severe drift. We study a mechanism that targets this setting through explicit operator composition. Holonomic Networks associate tokens with learned geometric actions in $SO(d)$ and implement computation by their ordered composition (holonomy). We formalize the Holonomic Network variable-binding swap-program instantiation in this operator language and show that the same compositional engine supports multiple decoders. Replacing variable readout by an identity-energy decoder yields a unified verifier for algebraic word identity (whether a sequence of generators evaluates to the identity element), scoring a word by the normalized Frobenius distance of its composed operator to the identity. We instantiate the template for finite Coxeter words in $S_{32}$ and infinite braid words in $B_8$, organizing specialization through a compact set of enforcement modes: hypothesis-class restriction to $SO(d)$, architectural tying where available (inverse-by-transpose), symmetry-alignment losses with anti-collapse guardrails, and oracle-audited evaluation. Trained only on short words ($L\le 50$), the resulting verifiers achieve perfect sampled classification under a $100\times$ length extrapolation to $L=5000$ in both settings. In the Coxeter case, identity and non-identity energies remain separated by a persistent positive gap, yielding $\mathrm{TPR}=1$ and $\mathrm{FPR}=0$ on the reported evaluations. For braid words, an independent SageMath audit confirms agreement with the oracle on a 14,000-example evaluation dump with zero detected label mismatches.
URL: https://openreview.net/forum?id=rsbDkbX8h0
---
Title: VT-DUDA: Visual Token Conditioning for Diffusion-guided Unsupervised Domain Adaptation
Abstract: Unsupervised domain adaptation (UDA) aims to learn a target-domain classifier from labeled source data and unlabeled target data under distribution shift. Recent diffusion-based UDA methods approach this problem by synthesizing labeled target-style images and training on the resulting synthetic data. However, their performance depends heavily on the conditioning design: class prompts provide only coarse guidance, while domain adaptation modules mainly control appearance, which may leave target-style synthesis insufficiently specified. We propose VT-DUDA, a visual-token conditioning framework for diffusion-guided UDA. Instead of relying only on text prompts, VT-DUDA uses source images to provide additional instance-level visual context for target-style synthesis. Specifically, it converts each source image into a compact set of visual tokens and injects them, together with text embeddings, into the standard cross-attention pipeline of a latent diffusion model. This provides instance-dependent conditioning beyond text alone, while synthesis is performed with the target-domain adapter branch. Because guidance is represented explicitly as a token sequence, the same interface also permits inference-time manipulation of the conditioning signal through token selection and token-strength adjustment. The proposed method preserves the standard diffusion objective and can be integrated into existing adapter-based diffusion frameworks without modifying the backbone. Across Office-31, Office-Home, and VisDA-2017, VT-DUDA improves average target-domain accuracy over strong discriminative and diffusion-based UDA baselines. The results suggest that, in generation-based UDA, a stronger conditioning interface can improve the downstream usefulness of synthetic target-style data.
URL: https://openreview.net/forum?id=Y956680PCe
---
Title: DecompRL: Solving More Problems with Less Tokens
Abstract: While repeated sampling from Large Language Models (LLMs) is a robust baseline for competitive programming and other automatically verifiable problems, it comes at a steep GPU cost. Reinforcement learning (RL)-based post-training can reduce the necessary sample size, but often worsens generations diversity, which limits performance in the large-scale sampling regime. Online RL is itself bottlenecked by the performance of the starting policy and the heavy compute required for inference. We introduce DecompRL, an algorithm inspired by modular inference that trains policies to decompose complex problems into separate, parallelizable functions. By recombining these modules into polynomially many solutions, DecompRL shifts the RL bottleneck from GPU-based inference to CPU-based evaluation. This enables massive scaling at a fraction of the cost, improving sparse reward discovery and solving complex problems that remain out of reach for standard RL.
URL: https://openreview.net/forum?id=RnBSdiMyBZ
---
Title: EdgeMask-HGNN: Learning to Sparsify Hypergraphs in Hypergraph Neural Networks
Abstract: Hypergraph Neural Networks (HGNNs) have achieved remarkable performance in various learning tasks involving hypergraphs. However, existing HGNNs consider the input hypergraph as fixed during training, ignoring the fact that real-world hypergraphs may often contain noisy hyperedges or links that are irrelevant for the downstream task. This makes them prone to overfitting, poor generalizability, and degrades their effectiveness on heterophilic hypergraphs. To address these issues, we propose EdgeMask-HGNN, a supervised sparsification method that learns a discrete subhypergraph under an explicit budget constraint. EdgeMask-HGNN offers two distinct sparsification schemes: a fine-grained sparsification and a coarse-grained sparsification, both trained end-to-end using supervision from the downstream task. Extensive experiments on node classification benchmarks demonstrate that EdgeMask-HGNN is effective on heterophilic hypergraphs. On more homophilic datasets, its performance is often comparable to strong baselines. Beyond node
classification, EdgeMask-HGNN also shows superior link prediction performance on existing link prediction benchmarks compared to full training and unsupervised sparsification baselines.
URL: https://openreview.net/forum?id=1AZijZObky
---
Title: Jacobian-aware Posterior Sampling for Inverse Problems
Abstract: Diffusion models provide powerful generative priors for solving inverse problems by sampling from a posterior distribution conditioned on corrupted measurements. Existing methods primarily follow two paradigms: direct methods, which approximate the likelihood term, and proximal methods, which incorporate intermediate solutions satisfying measurement constraints into the sampling process. We demonstrate that these approaches differ fundamentally in their treatment of the diffusion denoiser's Jacobian within the likelihood term. While this Jacobian encodes critical prior knowledge of the data distribution, training-induced non-idealities can degrade performance in zero-shot settings.
In this work, we bridge direct and proximal approaches by proposing a principled Jacobian-Aware Posterior Sampler (JAPS). JAPS leverages the Jacobian's prior knowledge while mitigating its detrimental effects through a corresponding proximal solution, requiring no additional computational cost. Our method enhances reconstruction quality across diverse linear and nonlinear noisy imaging tasks, outperforming existing diffusion-based baselines in perceptual quality while maintaining or improving distortion metrics.
URL: https://openreview.net/forum?id=m63GJnhIN2
---
Title: Multi-Constraint Online Convex Optimization with Adversarial Constraints
Abstract: We study online convex optimization with multiple adversarial constraints, where at each round a learner selects an action, and an adversary simultaneously reveals a convex cost function and $K$ convex constraint functions. The learner aims to minimize regret while keeping the cumulative constraint violation (CCV) of each individual constraint small. We introduce the Multi-Constraint Constrained Online Convex Optimization (MC-COCO) framework and develop a unified algorithmic approach based on exponential Lyapunov potentials. The key insight is that encoding all $K$ constraint violations via the potential $S_t = \sum_{k=1}^{K} e^{\lambda Q_k(t)}$ yields a surrogate cost whose growth ratio is controlled by the maximum single-round violation rather than the number of constraints $K$. This decoupling enables a per-constraint CCV of $\widetilde{O}(T^{1-\beta} \ln K)$, where $\beta \in [0,1]$ is a tunable regret-CCV trade-off parameter, improving qualitatively over the linear $K$-dependence of naive approaches. We instantiate the framework across three canonical settings (constrained experts, general Lipschitz-convex, and smooth convex) and further develop extensions for heterogeneous constraint prioritization (where critical constraints can be controlled at the $\widetilde{O}(T^{1-\beta}/\alpha_k)$ level) and long-term budget feasibility. Experiments on adversarial instances with up to $K=100$ constraints validate the theoretical bounds and confirm the logarithmic scaling in $K$.
URL: https://openreview.net/forum?id=3sLjLHCGzS
---
Title: Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learning
Abstract: Learning general-purpose reasoning capabilities has long been a challenging problem in AI. Recent research in LLMs, such as DeepSeek-R1, has shown that reinforcement learning techniques like GRPO enable pre-trained LLMs to develop reasoning capabilities using simple question-answer pairs. In this paper, we aim to train visual language models (VLMs) to perform reasoning on image data through reinforcement learning and visual question-answer pairs, without explicitly using any chain-of-thought (CoT) supervision. Our key finding indicates that simply applying GRPO to a VLM---by prompting the model to think step by step before giving an answer---may cause the model to develop shortcuts from easy questions, resulting in poor generalization of reasoning to broader question domains. We argue that the key to mitigating shortcut learning is to encourage the model to interpret images prior to reasoning. Therefore, we train the model to adhere to a caption-reason-answer output format: initially generating a detailed caption for an image, followed by constructing an extensive reasoning chain. When trained on 273K CoT-free visual question-answer pairs and using only reinforcement learning, our model, named Visionary-R1, outperforms strong multimodal models (e.g., GPT-4o, Claude3.5-Sonnet, and Gemini-1.5-Pro) on multiple visual reasoning benchmarks. Code and models will be open-sourced.
URL: https://openreview.net/forum?id=JWkZXBgh5a
---
Title: Semantic F1 Scores: Fair Evaluation Under Fuzzy Class Boundaries
Abstract: We propose Semantic F1 Scores, novel evaluation metrics for subjective or fuzzy multi-label classification that quantify semantic relatedness between predicted and gold labels. Unlike the conventional F1 metrics that treat semantically related predictions as complete failures, Semantic F1 incorporates a label similarity matrix to compute soft precision-like and recall-like scores, from which the Semantic F1 scores are derived. Unlike existing similarity-based metrics, our novel two-step precision-recall formulation enables the comparison of label sets of arbitrary sizes without discarding labels or forcing matches between dissimilar labels. By granting partial credit for semantically related but nonidentical labels, Semantic F1 better reflects the realities of domains marked by human disagreement or fuzzy category boundaries. In this way, it provides fairer evaluations: it recognizes that categories overlap, that annotators disagree, and that downstream decisions based on similar predictions lead to similar outcomes.
Through theoretical justification and extensive empirical validation on synthetic and real data, we show that Semantic F1 demonstrates greater interpretability and ecological validity. Because it requires only a domain-appropriate similarity matrix, which is robust to misspecification, and not a rigid ontology, it is applicable across tasks and modalities.
URL: https://openreview.net/forum?id=U0YJpGuFEc
---
Title: What Survives Privatization? A Guide to Structure and Utility in Differentially Private Genome-Wide Association Studies
Abstract: Single nucleotide polymorphisms (SNPs) are among the most common and informative forms of genetic variation in the human genome and constitute the primary data representation used in genome-wide association studies (GWAS). Due to their extreme dimensionality, strong correlation structure, and the presence of both population-level and familial dependencies, SNP datasets exhibit structural properties that fundamentally distinguish them from standard tabular data. At the same time, genomic data is uniquely sensitive; it is immutable, identifying, and shared across relatives, and has been shown to be vulnerable to a wide range of attacks, including membership inference, reconstruction, and kinship inference. As a result, protecting SNP data has become a critical and practically unavoidable requirement.
Differential privacy (DP) provides a rigorous mathematical framework for protecting sensitive data under strong adversarial assumptions. However, in the context of GWAS, the design and evaluation of meaningful DP mechanisms crucially depend on understanding the biological, statistical, and structural properties of SNP data and the downstream analysis pipelines. For a typical privacy researcher, acquiring even the minimal domain knowledge required to reason correctly about the structure of genomic data and the associated analysis pipelines represents a substantial and time-consuming barrier. Yet, without this understanding, progress in private genomic data analysis risks being misguided or misleading.
This survey explicitly bridges this gap. We provide a structured, self-contained primer on the structural properties of SNP data and the core analytical workflows of GWAS, focusing on the aspects most consequential for privacy definitions, mechanism design, and utility. Building on this foundation, we present the first comprehensive and systematic overview of differentially private methods for SNP datasets. We organize the literature through a release-oriented taxonomy that reframes existing approaches in terms of what survives privatization, revealing the design choices and trade-offs that shape their scientific and practical utility. Finally, we identify key open challenges arising from mismatches between existing differential privacy methodologies and the scientific, statistical, and operational realities of genomic data analysis, and outline future research directions toward principled and deployable privacy-preserving GWAS.
URL: https://openreview.net/forum?id=6BWikkmkOH
---
Title: Disentangling and Re-evaluating Similarity-Based Graph Structure Learning for GNNs on Node Classification
Abstract: Graph Structure Learning (GSL) is widely used to improve Graph Neural Networks (GNNs), especially through similarity-based graph construction for node classification. However, it remains unclear whether the reported gains come from the learned graph itself or from the node representations used to build that graph. In this paper, we study this question through a framework that decomposes GSL into three steps: (1) GSL-base generation (\ie processed node embeddings), (2) graph construction, and (3) multi-view fusion. Through empirical analysis and theoretical results, we show that, in the similarity-based setting, graph convolution on the constructed graph does not increase the Mutual Information (MI) between node representations and labels. This suggests that improvements often come from the quality of the GSL bases rather than from the graph construction procedure. To test this claim, we evaluate 450 GSL variants and compare them with GNN baselines under a shared search space of GSL bases. In this setting, similarity-based graph construction provides limited or inconsistent gains, whereas strong pre-trained GSL bases account for most of the improvement. These results clarify which components of GSL matter most for node classification and suggest that simpler GSL designs may be sufficient in many cases.
URL: https://openreview.net/forum?id=j7xOZTLIM0
---
Title: Can VLMs Reason Robustly? A Neuro-Symbolic Investigation
Abstract: Vision-Language Models (VLMs) have been applied to a wide range of reasoning tasks, yet it remains unclear whether they can reason robustly under distribution shifts. In this paper, we study covariate shifts in which the perceptual input distribution changes while the underlying prediction rules do not. To investigate this question, we consider visual deductive reasoning tasks, where a model is required to answer a query given an image and logical rules defined over the object concepts in the image. Empirically, we find that VLMs fine-tuned through gradient-based end-to-end training can achieve high in-distribution accuracy but fail to generalize under such shifts, suggesting that fine-tuning does not reliably induce the underlying reasoning function. This motivates a neuro-symbolic perspective that decouples perception from reasoning. However, we further observe that recent neuro-symbolic approaches that rely on black-box components for reasoning can still exhibit inconsistent robustness across tasks. To address this issue, we propose VLC, a neuro-symbolic method that combines VLM-based concept recognition with circuit-based symbolic reasoning. In particular, task rules are compiled into a symbolic program, specifically a circuit, which executes the rules exactly over the object concepts recognized by the VLM. Experiments on three visual deductive reasoning tasks with distinct rule sets show that VLC consistently achieves strong performance under covariate shifts, highlighting its ability to support robust reasoning.
URL: https://openreview.net/forum?id=4y6jiE6Q60
---
Title: GeoCache: Provably Lossless Privacy-Preserving KV-Cache Sharing in Multi-Tenant LLM Inference via Isometric Orthogonal Transformation
Abstract: Key-value (KV) cache sharing across user sessions is a critical optimization for multi- tenant large language model (LLM) serving, reducing time-to-first-token (TTFT) latency by up to 73%. However, recent work has demonstrated that shared KV caches create exploitable vulnerabilities: an adversary can exfiltrate data or mount semantic poisoning attacks by accessing cross-tenant cache states. Existing mitigations either disable sharing entirely (negating performance gains) or employ heuristic content classifiers with inherent false-positive/negative tradeoffs. We propose GeoCache, a principled approach that ap- plies session-specific orthogonal transformations to key and value attention matrices before caching. GeoCache exploits a fundamental property of orthogonal operators — isometry — to provide two simultaneous guarantees: (1) exact preservation of attention computation for authorized in-session cache reuse, proven by the identity q⊤M⊤MK = q⊤K; and (2) geometric isolation of cross-session cache entries, with attention scores concentrating at zero with probability at least 1−2exp(−dkτ2/2). The approach uses a Block-Diagonal Orthogo- nal Transform (BDOT) achieving O(dk) per-token overhead and sub-microsecond (0.014μs) latency per head-token on an NVIDIA T4 GPU. Experiments on Llama-2-7B and Mistral- 7B demonstrate that GeoCache preserves model output quality exactly (mathematical equivalence with maximum float32 deviation 5.3 × 10−5), makes cross-session cache entries indistinguishable from random noise (rendering data exfiltration and semantic poisoning mathematically infeasible), and introduces negligible throughput overhead. GeoCache pro- vides mathematically guaranteed cross-session isolation with provably zero inference quality degradation.
URL: https://openreview.net/forum?id=CNuH7VtNun
---
Title: Multiscale Co-Manifold Learning on Tensors
Abstract: Nonlinear manifold learning on tensors typically compute mode-wise embeddings that fail to capture implicit couplings between the geometries of the different modes. We propose a new framework called tensor co-manifold learning (TCML). TCML is designed to recover coupled low-dimensional structures {\em simultaneously} across all modes of multiway data (i.e., tensors or multi-dimensional arrays) and generalizes recent methods for co-manifold learning to higher-order tensors via a tensor-based multiscale approach to co-organizing rows, columns, and higher modes. By imposing smoothness constraints at various levels of granularity, we formulate a family of optimization problems that characterize smoothness across coarse-to-fine scales. We demonstrate that these problems are efficiently solvable and their solutions yield a multiscale distance between tensor slices along a given mode. These distances take into account the structure of the data along the other modes. We demonstrate how to utilize this multiscale distance measure to compute nonlinear embeddings of the data. The resulting embeddings are demonstrably more effective at revealing low-dimensional coupled structure than linear factorizations or nonlinear embeddings obtained by treating each mode independently.
URL: https://openreview.net/forum?id=JA5bfaiaaa
---
Title: Hallucination Detection and Mitigation with Diffusion in Multi-Variate Time-Series Foundation Models
Abstract: Foundation models (FMs) for natural language processing have many coherent definitions of hallucination and methods for its detection and mitigation. However, analogous definitions and methods do not exist for multi-variate time-series (MVTS) FMs. We propose new definitions for MVTS hallucination, along with new detection and mitigation methods using a diffusion model to estimate hallucination levels. We derive relational datasets from popular time-series datasets to benchmark these relational hallucination levels. Using these definitions and models, we find that open-source pre-trained MVTS imputation FMs relationally hallucinate on average up to 59.5\% as much as a weak baseline. The proposed mitigation method reduces this by up to 47.7\% for these models. The definition and methods may improve adoption and safe usage of MVTS FMs.
URL: https://openreview.net/forum?id=fHGQ7hZlb5
---
Title: X-KB-UCB: Decentralized Cross Kriging-Believer UCB for Static and Time-Varying GP Bandits
Abstract: We study decentralized Gaussian process (GP) bandits under strict communication budgets and a shared reward function. We introduce X-KB-UCB, a gossip-based Upper-Confidence Bound (UCB) method in which agents periodically exchange only their most recent chosen arm and the observed reward. At gossip rounds, agents coordinate exploration through a cross-agent Kriging--Believer update, while between gossip rounds each agent follows the corresponding single-agent rule, GP-UCB for static rewards and TV-GP-UCB for time-varying rewards. We provide high-probability no-regret guarantees for augmented agents, using an agent-centric accounting that includes both locally collected and gossiped observations, in both the static setting and a time-varying setting modeled by a Markov-drift GP. The resulting bounds are expressed in terms of information gain and recover standard single-agent rates when gossip is absent. In the always-gossip regime, they match the centralized batch-selection rate of GP-BUCB, with an additional term reflecting drift. Experiments confirm that gossip yields consistent gains over independent agents and approaches a centralized baseline under the same evaluation budget.
URL: https://openreview.net/forum?id=7jHmCJa9QF
---
Title: Towards Representation Backdoor on CLIP via Concept Confusion
Abstract: Backdoor attacks pose a serious threat to deep learning models by allowing adversaries to implant hidden behaviors that remain dormant on clean inputs but are maliciously triggered at inference. Existing backdoor attack methods typically rely on explicit triggers such as image patches or pixel perturbations, which makes them easier to detect and limits their applicability in complex settings. To address this limitation, we take a different perspective by analyzing backdoor attacks through the lens of concept-level reasoning, drawing on insights from interpretable AI. We show that traditional attacks can be viewed as implicitly manipulating the concepts activated within a model’s latent space. This motivates a natural question: can backdoors be built by directly manipulating concepts? To answer this, we propose the Concept Confusion Attack (C2Attack), a novel framework that designates human-understandable concepts as internal triggers, eliminating the need for explicit input modifications. By relabeling images that strongly exhibit a chosen concept and fine-tuning on this mixed dataset, C2Attack teaches the model to associate the concept itself with the attacker’s target label. Consequently, the presence of the concept alone is sufficient to activate the backdoor, making the attack stealthier and more resistant to existing defenses. Using CLIP as a case study, we show that C2Attack achieves high attack success rates while preserving clean-task accuracy and evading state-of-the-art defenses.
URL: https://openreview.net/forum?id=jQ91DETUIr
---
Title: TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generation
Abstract: When generating images from prompts that include specific entities, the model must retain as much entity-specific knowledge as possible.
However, the number of entities is enormous, and new entities emerge; memorizing all of them completely is not realistic.
To bridge this gap, our work proposes Text-based Intelligent Generation with Entity Prompt Refinement (\textsc{TextTIGER}).
\textsc{TextTIGER} strengthens knowledge about entities that appear in the prompt by augmenting external information and then summarizes the expanded descriptions with large language models, preventing performance degradation that arises from excessively long inputs.
To evaluate our method, we construct a new dataset consisting of captions, images, detailed descriptions, and lists of entities.
Experiments with multiple image generation models show that \textsc{TextTIGER} improves image generation performance on widely used evaluation metrics compared with prompts that use captions alone.
In addition, using Multimodal LLM (MLLM)-as-a-judge, which shows a strong correlation with human evaluation, we demonstrate that our method consistently achieves higher scores, which underscores its effectiveness.
These results show that strengthening entity-related descriptions, summarizing them, and refining prompts to an appropriate length leads to substantial improvements in image generation performance.
We will release the created dataset and code upon acceptance.
URL: https://openreview.net/forum?id=jI6s4EsbRS
---
Title: Wasserstein-type Gaussian Process Regressions for Input Measurement Uncertainty
Abstract: Gaussian process (GP) regression is widely used for uncertainty quantification, yet the standard formulation assumes noise-free covariates. When inputs are measured with error, this errors-in-variables (EIV) setting can lead to optimistically narrow posterior intervals and biased decisions. We study GP regression under input measurement uncertainty by representing each noisy input as a probability measure and defining covariance through Wasserstein distances between these measures. Building on this perspective, we instantiate a deterministic projected Wasserstein ARD (PWA) kernel whose one-dimensional components admit closed-form expressions and whose product structure yields a scalable, positive-definite kernel on distributions. Unlike latent-input GP models, PWA-based GPs (\PWAGPs) handle input noise without introducing unobserved covariates or Monte Carlo projections, making uncertainty quantification more transparent and robust.
URL: https://openreview.net/forum?id=yEZjju9CUw
---
Title: From Arithmetic to Logic: The Resilience of Logic and Lookup-Based Neural Networks Under Parameter Bit-Flips
Abstract: The deployment of Deep Neural Networks (DNNs) in safety-critical edge environments necessitates robustness against hardware-induced bit-flip errors. While empirical studies indicate that reducing Numerical Precision can improve fault tolerance, the theoretical basis of this phenomenon remains underexplored. In this work, we study resilience as a Structural Property of neural architectures rather than solely as a property of a dataset-specific trained solution. By deriving the Expected Squared Error (MSE) under independent parameter bit flips across multiple numerical formats and layer primitives, we show that lower precision, higher sparsity, bounded activations, and shallow depth are consistently favored under this corruption model. We then argue that Logic and Lookup-Based Neural Networks realize the joint limit of these design trends. Through ablation studies on the MLPerf Tiny benchmark suite, we show that the observed empirical trends are consistent with the theoretical predictions, and that LUT-based models remain highly stable in corruption regimes where standard floating-point models fail sharply. Furthermore, we identify a novel Even-Layer Recovery effect unique to logic-based architectures and analyze the structural conditions under which it emerges. Overall, our results suggest that shifting from continuous arithmetic weights to discrete Boolean lookups can provide a favorable Accuracy--Resilience trade-off for hardware fault tolerance.
URL: https://openreview.net/forum?id=ZZYvGZei5h
---
Title: Randomized Asymmetric Chain of LoRA: The First Optimization-Theoretic Framework for Low-Rank Adaptation
Abstract: Fine-tuning is a common approach to adapt large models to specific tasks. Low-Rank Adaptation (LoRA) is a widely used method for parameter-efficient fine-tuning, where updates are represented as products of low-rank matrices. Although LoRA performs well in practice, it often gives weaker results than full-parameter fine-tuning (FPFT), and its theoretical understanding is still limited. In this work, we show that LoRA and its extensions, such as Asymmetric LoRA and Chain of LoRA, can have convergence problems. To solve this, we introduce Randomized Asymmetric Chain of LoRA (RAC-LoRA), a general optimization framework that provides theoretical guarantees of convergence for LoRA-based methods. Our approach keeps the practical advantages of LoRA but adds important algorithmic changes to ensure convergence. We prove that RAC-LoRA can reach the same solution as FPFT, and we analyze its convergence rate. Our results cover smooth, non-convex objectives and include gradient descent, stochastic gradient descent, and federated learning setups. Experiments support our theory.
URL: https://openreview.net/forum?id=Y1TOtwtrjS
---
Title: Adaptive Federated Learning via Dynamical System Model
Abstract: Hyperparameter selection is critical for stable and efficient convergence of heterogeneous federated learning, where clients differ in computational capabilities, and data distributions are non-IID. Tuning hyperparameters is a manual and computationally expensive process as the hyperparameter space grows combinatorially with the number of clients. To address this, we introduce an end-to-end adaptive federated learning method in which both clients and central agents adaptively select their local learning rates and momentum parameters.
Our approach models federated learning as a dynamical system, allowing us to draw on principles from numerical simulation and physical design. Through this perspective, selecting momentum parameters equates to critically damping the system for fast, stable convergence, while learning rates for clients and central servers are adaptively selected to satisfy accuracy properties from numerical simulation. The result is an adaptive, momentum-based federated learning algorithm in which the learning rates for clients and servers are dynamically adjusted and controlled by a single, global hyperparameter. By designing a fully integrated solution for both adaptive client updates and central agent aggregation, our method is capable of handling key challenges of heterogeneous federated learning, including objective inconsistency and client drift. Importantly, our approach achieves fast convergence while being insensitive to the choice of the global hyperparameter, making it well-suited for rapid prototyping and scalable deployment. Compared to state-of-the-art adaptive methods, our framework is shown to deliver superior convergence for heterogeneous federated learning while eliminating the need for hyperparameter tuning both client and server updates.
URL: https://openreview.net/forum?id=ebDfXAklXg
---
Title: A Survey of Flow Matching in Reinforcement Learning
Abstract: Flow Matching (FM) has recently emerged as a principled and efficient generative modeling framework for reinforcement learning (RL), enabling expressive, multimodal policy parameterizations via deterministic probability transport.
Compared to diffusion-based policies that rely on stochastic denoising chains, FM uses sampling based on ordinary differential equations (ODEs), with learned velocity fields, which can substantially reduce inference latency and simplify the incorporation of RL objectives.
As research in flow-based RL rapidly accelerates across offline continuous control, online fine-tuning, and foundation model alignment, the literature has become highly fragmented. In this survey, we provide the a comprehensive taxonomy of flow-matching approaches in reinforcement learning. We organize the literature along two axes: the target distribution being modeled (e.g., action policies, value critics, transition dynamics) and the mechanism of RL signal integration (e.g., energy-weighted regression, flow-based policy gradients, and group relative policy optimization).
Furthermore, we survey emerging frontiers such as discrete and non-Euclidean action spaces, provide a systematic comparative analysis against Gaussian and diffusion baselines, and outline critical open problems. Ultimately, this survey serves as a foundational roadmap for the next generation of generative reinforcement learning and alignment.
URL: https://openreview.net/forum?id=P6e5IC4gPe
---
Title: Coupled Distributional Random Expert Distillation for World Model Online Imitation Learning
Abstract: Imitation Learning (IL) has achieved remarkable success across various domains, including robotics, autonomous driving, and healthcare, by enabling agents to learn complex behaviors from expert demonstrations. However, existing IL methods often face instability challenges, particularly when relying on adversarial reward or value formulations in world model frameworks. In this work, we propose a novel approach to online imitation learning that addresses these limitations through a reward model based on random network distillation (RND) for density estimation. Our reward model is built on the joint estimation of expert and behavioral distributions within the latent space of the world model. We evaluate our method across diverse benchmarks, including DMControl, Meta-World, and ManiSkill2, showcasing its ability to deliver stable performance and achieve expert-level results in both locomotion and manipulation tasks. Our approach demonstrates improved stability over adversarial methods while maintaining expert-level performance.
URL: https://openreview.net/forum?id=3cHzUD6Sx3
---
Title: DIMENSION DOMAIN CO-DECOMPOSITION: SOLVING PDES WITH INTERPRETABILITY
Abstract: Physics-informed neural networks (PINNs) have demonstrated effectiveness in solving partial differential equations (PDEs), yet they often struggle in high-dimensional regimes and lack interpretable representations and in scenarios involving sharp solution structures. Moreover, existing approaches typically rely on manually specified domain partitions. We propose a unified Dimension–Domain Co-Decomposition (3D) framework that jointly integrates dimension-wise decomposition with mixture-of-experts (MoE)–based domain decomposition. At the dimension level, we introduce an interpretable decomposition mechanism in which coordinate inputs are decoupled within each expert through a shared MLP with indexed inputs, enabling parameter efficiency while preserving expressivity. To quantitatively assess interpretability, we define a Variable Interpretability (VI) metric that measures the alignment between learned latent components and the corresponding solution factors. At the domain level, an MoE-based gating mechanism adaptively partitions the solution space without requiring predefined regions or interface conditions. Extensive experiments on PDE benchmarks demonstrate that the proposed framework achieves improved accuracy and computational efficiency compared to standard PINNs and related baselines, while providing interpretable and scalable representations.
URL: https://openreview.net/forum?id=kuzkynVyRq
---
Title: Realistic Evaluation of Model Merging for Compositional Generalization
Abstract: Model merging has emerged as a practical and cost-effective approach for combining multiple pretrained models into a single model that inherits their capabilities and often achieves improved performance. Its growing popularity has led to the rapid development of numerous merging techniques. However, these methods are typically evaluated in disparate experimental settings and make differing assumptions about model architecture, data availability, and computational budget, making direct comparison difficult. In this work, we systematically characterize the relative strengths and limitations of existing merging methods by evaluating them within a unified experimental framework. Our study focuses on compositional generalization --- \ie whether merging can successfully combine distinct skills to generalize to new settings. We also analyze the computational costs of each method and examine how performance scales as the number of merged models increases. Overall, we evaluate eight merging methods in a novel benchmark spanning three distinct cross-modal settings, resulting in 12,000 unique merge configurations. Our findings reveal the absence of a one-size-fits-all merging strategy and serves as both an outline for the holistic evaluation of future merging methods as well as a cookbook for practitioners using model merging.
URL: https://openreview.net/forum?id=j7ye0nXvEm
---
Title: Transform-Triggered Adversarial Examples
Abstract: Deep neural networks are vulnerable to adversarial attacks, yet most existing attack research focuses on adversarial examples that induce fixed, static mispredictions. In this work, we instead exploit a dynamical adversarial manifold that depends on image transforms, which are a group of functions commonly used for data augmentation, preprocessing, and deployment. We incorporate image transforms into the adversarial optimization process, such that at test-time the same transforms, when applied under malicious conditions, act as triggers that induce diverse adversarial behaviors. For the first time, we demonstrate adversarial perturbations that exhibit metamorphic properties, producing different attack outcomes under different transforms. Our study shows that this transform-dependent vulnerability consistently exists across multiple deep network architectures (e.g., CNNs and transformers), computer vision tasks (e.g., image classification and object detection), and a broad range of commonly used image transforms. Additionally, to further motivate its real-world relevance, we extend our transform-dependent formulation to a camera-in-the-loop setting, demonstrating its effectiveness under challenging physical conditions. In summary, we introduce a novel and controllable paradigm for adversarial attack deployment, exposing a previously overlooked and structurally inherent vulnerability in deep neural networks.
URL: https://openreview.net/forum?id=If2OGZMJxs
---
Title: Discovering Generalizable Governing Equations for Graph Dynamical Systems with Interpretable Neural Networks
Abstract: The discovery of symbolic governing equations is a central goal in science; yet, it remains challenging particularly for graph dynamical systems, where the network topology further shapes the system behavior. While artificial intelligence offers powerful tools for modeling these dynamics, the field lacks a rigorous comparative benchmark to assess the true scientific utility of the discovered laws. To address this challenge, this work proposes a novel evaluation pipeline designed to rigorously assess state-of-the-art symbolic regression models for graph equation discovery. Moving beyond simple fitting metrics, this framework evaluates discovered laws based on their long-term trajectory stability and, critically, their out-of-distribution generalization to unseen graph topologies. We benchmark established methods, including sparse regression and MLP-based architectures, and introduce the Graph Kolmogorov-Arnold Network-ODE (GKAN-ODE) model, a novel adaptation of KANs explicitly tailored for this domain, augmented by hyperparameter-free multiplicative nodes and a new Spline-Wise symbolic regression algorithm. Across a suite of synthetic and real-world graph dynamical systems, we numerically demonstrate through extensive experiments that neural-based approaches, particularly the GKAN-ODE model, recover exact ground-truth equations and achieve trajectory errors up to two orders of magnitude lower than the baseline methods on out-of-distribution test graphs.
URL: https://openreview.net/forum?id=a2mPNSSAYL
---
Title: PrIntMesh: Precise Intersection Surfaces for 3D Organ Mesh Reconstruction
Abstract: Human organs are composed of interconnected substructures whose geometry and spatial relationships constrain one another. Yet, most deep-learning approaches treat these parts independently, producing anatomically implausible reconstructions. We introduce PrIntMesh, a template-based, topology-preserving framework that reconstructs organs as unified systems. Starting from a connected template, PrIntMesh jointly deforms all substructures to match patient-specific anatomy, while explicitly preserving internal boundaries and enforcing smooth, artifact-free surfaces. We demonstrate its effectiveness on the heart, hippocampus, and lungs, achieving high geometric accuracy, correct topology, and robust performance even with limited or noisy training data. Compared to voxel- and surface-based methods, PrIntMesh better reconstructs shared interfaces, maintains structural consistency, and provides a data-efficient solution suitable for clinical use.
URL: https://openreview.net/forum?id=httuGMEO9Q
---
Title: The Fake Mirror Effect: Foreign Feedback Disrupts Self-Correction in Minimal Recurrent Networks
Abstract: We construct a minimal recurrent network (35 neurons) and mechanistically dissect how it learns to correct predictions through output feedback. In a variable-noise task, the network achieves a correction gain of +0.119 (N=20, 95% CI excluding zero) by accumulating evidence across timesteps; ablating the recurrent loop eliminates this gain. Replacing a model’s own feedback with another independently trained model’s output ("clone feedback") degrades performance below the no-feedback baseline at the tested minimal scale — even when the clone’s output has valid statistics. This "fake-mirror" effect is graded: self-feedback > same-model-wrong-trial feedback (positive) > clone and shuffled feedback (negative), with feedback interpolation revealing smooth monotonic degradation. Progressively stronger static aligners (30-17,925 parameters) recover most of the clone-feedback loss but plateau at ~85% recovery, indicating that, within the tested alignment family, open-loop mapping is insufficient to fully restore closed-loop compatibility. Parameter-matched and compute-matched feedforward controls produce zero gain, and the core findings replicate on MNIST (120k parameters), though geometric mismatch harm attenuates at larger scales. These results provide evidence for *feedback-contract specificity*: the recurrent weights are co-adapted to the model’s own output geometry, and the fidelity of this alignment — not merely the presence of feedback — determines whether iterative refinement succeeds.
URL: https://openreview.net/forum?id=cwENvGCLRv
---
Title: Encoding Without Influence: Dissociating Demographic Representation from Causal Effect in Large Language Models
Abstract: Large language models are increasingly deployed in settings that require normative judg-
ment, yet the internal pathway by which demographic context shapes their outputs remains
uncharacterized. We apply sparse autoencoder feature extraction and causal interventions
(activation patching, feature steering, and targeted ablation) to Gemma 2 9B, Qwen 2.5
7B, and Llama 3.1 8B, tracing how demographic information is represented and used dur-
ing survey responses across five policy domains. We find that demographic representations
and demographic influence are localized in different parts of the network: early layers en-
code demographic identity but exert no measurable effect on outputs, while interventions on
late-layer features recover 68.7–75.8% of behavioral effects across architectures. Variance-
matched null baselines confirm that these effects are specific to demographic features rather
than a generic consequence of perturbation. We further show that demographic influence
is domain-modulated, with the ranking of influential demographics shifting across policy
areas. The dissociation generalizes across three architectures with different encoding pro-
files and alignment procedures. These results suggest that representational detection alone
is insufficient for bias auditing, as the most detectable demographic encodings are not the
ones driving outputs, and that fairness evaluation must be both causally validated and
domain-specific.
URL: https://openreview.net/forum?id=TQbXHsI3Lm
---
Title: Lost in Aggregation: On a Fundamental Expressivity Limit of Message-Passing Graph Neural Networks
Abstract: We define a generic class of functions that captures most conceivable aggregations for Message-Passing Graph Neural Networks (MP-GNNs), and prove that any MP-GNN model with such aggregations induces only a polynomial number of equivalence classes on all graphs - while the number of non-isomorphic graphs is doubly-exponential (in number of vertices).
Adding a familiar perspective, we observe that merely 2-iterations of Color Refinement (CR) induce at least an exponential number of equivalence classes, making the aforementioned MP-GNNs relatively infinitely weaker. Previous results state that MP-GNNs match full CR, however they concern a weak, 'non-uniform', notion of distinguishing-power where each graph size may required a different MP-GNN to distinguish graphs up to that size.
Our results concern both distinguishing between non-equivariant vertices and distinguishing between non-isomorphic graphs.
URL: https://openreview.net/forum?id=F4OlpAJ0o2
---
Title: Trade-offs in Ensembling, Merging and Routing Among Parameter-Efficient Experts
Abstract: While large language models (LLMs) fine-tuned with lightweight adapters achieve strong performance across diverse tasks, their performance on individual tasks depends on the fine-tuning strategy. Fusing independently trained models with different strengths has shown promise for multi-task learning through three main strategies: ensembling, which combines outputs from independent models; merging, which fuses model weights via parameter averaging; and routing, which integrates models in an input-dependent fashion. However, many design decisions in these approaches remain understudied, and the relative benefits of more sophisticated ensembling, merging and routing techniques are not fully understood. We empirically evaluate their trade-offs, addressing two key questions: What are the advantages of going beyond uniform ensembling or merging? And does the flexibility of routing justify its complexity? Our findings indicate that non-uniform ensembling and merging improve performance, but routing offers even greater gains. To mitigate the computational cost of routing, we analyze expert selection techniques, showing that clustering and greedy subset selection can maintain reasonable performance with minimal overhead. These insights advance our understanding of model fusion for multi-task learning.
URL: https://openreview.net/forum?id=bnRCvRtZv5
---
Title: Yose-Ue: A Treap-Based Ensemble Framework for Resource-Efficient Unsupervised Anomaly Detection
Abstract: Anomaly detection seeks to identify observations that deviate significantly from an underlying data distribution. While deep learning and ensemble-based approaches have achieved strong empirical performance, their computational and memory requirements limit their applicability in resource-constrained edge environments. Furthermore, many approaches to improving efficiency rely on supervised models, which require labeled anomalies that are often scarce in practice. We propose Yose-Ue, a resource-efficient, fully unsupervised anomaly detection framework based on treap-structured ensemble learning. Yose-Ue co-designs a compact data representation with computationally efficient split-selection mechanisms. Specifically, we construct a randomized treap (a hybrid tree–heap data structure) in which nodes are defined by discretized split points, and priorities are assigned via a mass-driven criterion that favors informative partitions. This design yields balanced hierarchical partitions while maintaining low memory overhead. The resulting ensemble estimator improves structural diversity and statistical robustness without incurring the computational cost typical of deep architectures.
We provide a comparative evaluation against established unsupervised ensemble baselines (Isolation Forest, DiForest, and EXTiForest) and resource-efficient state-of-the-art methods, including AutoEncoder, Graph Attention Network AutoEncoder, Histogram-Based Outlier Score, and Local Outlier Factor. Experiments conducted on 14 benchmark datasets—including synthetic datasets, experimental mobile-sensor data, and datasets from the ODDS repository—demonstrate that Yose-Ue achieves competitive or superior detection performance while substantially reducing computational complexity. The proposed method attains over 126× reduction in training time and 7× reduction in inference latency relative to representative baselines. These results indicate that treap-based ensemble learning provides a principled and scalable approach to unsupervised anomaly detection in edge-constrained environments.
URL: https://openreview.net/forum?id=17y2ooyemG
---
Title: What do vision-language models see (or not) in the context? Investigating multimodal in-context learning
Abstract: In-context learning (ICL) enables Large Language Models (LLMs) to learn tasks from demonstration examples without parameter updates. Although it has been extensively studied in LLMs, its effectiveness in Vision-Language Models (VLMs) remains underexplored. In this work, we present a systematic study of ICL in VLMs, evaluating seven models spanning four architectures on three image captioning benchmarks. We analyze how prompt design, architectural choices, and training strategies influence multimodal ICL. To our knowledge, we are the first to analyze how attention patterns in VLMs vary with an increasing number of in-context demonstrations. Our results reveal that training on image–text interleaved data enhances ICL performance but does not imply effective integration of visual and textual information from demonstration examples. In contrast, instruction tuning improves instruction-following but can reduce reliance on in-context demonstrations, suggesting a trade-off between instruction alignment and in-context adaptation. Attention analyses further show that current VLMs primarily focus on textual cues and fail to leverage visual information, suggesting a limited capacity for multimodal integration. These findings highlight key limitations in the ICL abilities of current VLMs and provide insights for enhancing their ability to learn from multimodal in-context examples.
URL: https://openreview.net/forum?id=Gl3FlNwJP6
---
Title: End-to-End 4D Heart Mesh Recovery Across Full-Stack and Sparse Cardiac MRI
Abstract: Reconstructing cardiac motion from CMR sequences is critical for diagnosis, prognosis, and intervention. Existing methods rely on complete CMR stacks to infer full heart motion, limiting their applicability during intervention when only sparse observations are available.
We present TetHeart, the first end-to-end framework for unified 4D heart mesh recovery from both offline full-stack and intra-procedural sparse-slice observations.
Our method leverages deformable tetrahedra to capture shape and motion in a coherent space shared across cardiac structures. Before a procedure, it initializes detailed, patient-specific heart meshes from high-quality full stacks, which can then be updated using whatever slices can be obtained in real-time, down to a single one during the procedure.
TetHeart incorporates several key innovations: (i) an attentive slice-adaptive 2D–3D feature assembly mechanism that integrates information from arbitrary numbers of slices at any position; (ii) a distillation strategy to ensure accurate reconstruction under extreme sparsity; and (iii) a weakly supervised motion learning scheme requiring annotations only at keyframes, such as the end-diastolic and end-systolic phases. Trained and validated on three large public datasets and evaluated on additional private interventional and public datasets without retraining, TetHeart achieves state-of-the-art accuracy in both pre- and intra-procedural settings.
URL: https://openreview.net/forum?id=9k00kN5yk2
---
Title: Sensitivity of Small Language Models to Fine-tuning Data Contamination
Abstract: Small Language Models (SLMs) are increasingly being deployed in resource-constrained environments, yet their robustness to data contamination during instruction tuning remains poorly understood. We systematically investigate the contamination sensitivity of 23 SLMs (270M to 4B parameters) across different model families by measuring susceptibility to syntactic transformations (character and word reversal) and semantic transformations (irrelevant and counterfactual responses), each applied at contamination levels from 1% to 100%. Our results reveal fundamental asymmetries in vulnerability patterns, as syntactic transformations cause catastrophic performance degradation with character reversal producing near-complete failure across all models regardless of size or family, whereas semantic transformations demonstrate distinct threshold behaviors and greater resilience in core linguistic capabilities. We discover a 'capability curse' where larger, more capable models become more susceptible to learning semantic corruptions, effectively following harmful instructions, while our analysis of base versus instruction-tuned variants reveals that alignment provides inconsistent robustness benefits, sometimes even reducing resilience. Layerwise representational analysis across model families and sizes shows a consistent localization of contamination effects toward the final blocks, with syntactic corruption typically inducing stronger late-layer divergence and semantic corruption producing comparatively smaller changes that are often confined to final layers. Our work makes three contributions: (1) empirical evidence that SLMs are disproportionately vulnerable to syntactic contamination patterns, (2) characterization of asymmetric learning dynamics for syntactic versus semantic contamination supported by behavioral and representational analysis, and (3) systematic evaluation protocols for robustness assessment. These findings have deployment implications, suggesting that current robustness assumptions may not hold for smaller models and highlighting the need for contamination-aware training protocols that target late layer representations.
URL: https://openreview.net/forum?id=dIAL8bckQu
---
Title: Reconsidering Degeneration of Token Embeddings with Definitions
Abstract: While learning token embeddings via language modeling and weight tying remains the dominant paradigm, embeddings often degenerate into anisotropic (i.e., non-uniform) distributions in geometry space, limiting their expressiveness. This study first analyzes the fine-tuning dynamics of encoder-based pretrained language models (PLMs) and shows that their embeddings can largely preserve their geometric structure to defend against degeneration during fine-tuning. However, pretrained embeddings still suffer from anisotropic distribution, and low-frequency tokens tend to lose their semantics. To address this issue, we propose DefinitionEMB, a method that leverages lexical definitions to inject explicit semantics into embeddings while anchoring them to the pretrained geometric manifold to preserve PLMs' established geometric knowledge. Extensive experiments demonstrate the effectiveness of leveraging Wiktionary definitions on four PLMs: RoBERTa-base, BART-large, T5-large, and T5Gemma-l-ul2 across natural language understanding and abstractive text summarization.
URL: https://openreview.net/forum?id=3liQS0NptI
---
Title: Overcoming label shift in target-aware federated learning
Abstract: Federated learning enables multiple actors to collaboratively train models without sharing private data. Existing algorithms are successful and well-justified in this task when the intended target domain, where the trained model will be used, shares data distribution with the aggregate of clients, but this is often violated in practice. A common reason is label shift—that the label distributions differ between clients and the target domain. We demonstrate empirically that this can significantly degrade performance. To address this problem, we propose FedPALS, a principled and practical model aggregation scheme that adapts to label shifts to improve performance in the target domain by leveraging knowledge of label distributions at the central server. Our approach ensures unbiased updates under federated stochastic gradient descent, which yields robust generalization across clients with diverse, label-shifted data. Extensive experiments on image classification tasks demonstrate that FedPALS consistently outperforms baselines by aligning model aggregation with the target domain. Our findings reveal that conventional federated learning methods suffer severely in cases of extreme label sparsity on clients, highlighting the critical need for target-aware aggregation as offered by FedPALS.
URL: https://openreview.net/forum?id=dQAsAmb1Xb
---
Title: Client-Level Defense Placement for Adversarially Robust Federated Reinforcement Learning
Abstract: Federated Reinforcement Learning (FRL) extends federated learning to sequential decision-making, enabling multiple clients to collaboratively train a global policy without sharing raw trajectories. While this setting is promising for privacy-sensitive domains such as autonomous systems and IoT control, it introduces critical attack surfaces: adversaries can corrupt policy gradients, and adaptive attackers that reshuffle targets and prioritize high-impact clients render static defenses brittle. Defenses in FRL operate at two complementary layers: server-side aggregation and client-level placement, but the latter remains under-formalized despite directly shaping attacker incentives.
We propose FRL-CDPS (\textbf{C}lient-Level \textbf{D}efense \textbf{P}lacement for Adversarially Robust \textbf{F}ederated \textbf{R}einforcement \textbf{L}earning: A \textbf{S}tackelberg Approach), which models budget-constrained client-level defense placement as a Stackelberg game: the defender commits to a protection strategy while a rational Bayesian attacker best-responds under imperfect reconnaissance, maintaining posterior beliefs over each client's defense status. The framework captures partial observability and probabilistic defense effectiveness, faithfully reflecting real-world conditions where defenses are imperfect and adversaries operate under uncertainty. Despite NP-hardness of the defender's bilevel problem, we provide tractable solvers, namely exact feasible-set search for small systems and candidate-based Monte Carlo search for larger ones, with a $\frac{1}{2}$-approximation guarantee for the attacker oracle.
Experiments on CartPole-v1 and HalfCheetah-v2 across seven ablation dimensions show that FRL-CDPS consistently outperforms heuristic client-selection baselines (random, UCB, Thompson sampling) and composes effectively with server-side defenses (FLTG, FedGreed), demonstrating that Stackelberg planning provides a principled and practical advantage for client-level defense in FRL.
URL: https://openreview.net/forum?id=JRpLJhvSCY
---
Title: EdgeCrafter: Compact ViTs for Edge Dense Prediction via Task-Specialized Distillation
Abstract: Deploying high performance dense prediction models on resource-constrained edge devices remains challenging due to strict limits on computation and memory. In practice, lightweight systems for object detection, instance segmentation, and pose estimation are still dominated by CNN-based architectures such as YOLO, while compact Vision Transformers (ViTs) often struggle to achieve similarly strong accuracy–efficiency trade-offs, even with large scale pretraining. We argue that this gap is largely due to insufficient task-specific representation learning in small-scale ViTs, rather than an inherent mismatch between ViTs and edge dense prediction. To address this issue, we introduce EdgeCrafter, a unified compact ViT framework for edge dense prediction centered on ECDet, a detection model built from a distilled compact backbone and an edge-friendly encoder–decoder design. We first adapt a large DINOv3 pretrained ViT to object detection and use it as a task-specialized teacher to distill rich representations into compact student backbones. We further improve efficiency by replacing standard patch embedding with a lightweight convolutional stem and constructing multi-scale features with simple interpolation and linear projection instead of costly feature pyramids. The resulting detection-distilled representation transfers directly to instance segmentation and human pose estimation through lightweight task-specific prediction modules. On the COCO dataset, ECDet-S achieves 51.7 AP with fewer than 10M parameters using only COCO annotations. For instance segmentation, ECInsSeg achieves performance comparable to RF-DETR-Seg while using substantially fewer parameters and without the need for additional Objects365 pretraining. For pose estimation, ECPose-X reaches 74.8 AP, significantly outperforming YOLO26-Pose-X (71.6 AP). These results show that compact ViTs, when paired with task-specialized distillation and edge-aware design, can be a practical and competitive option for edge dense prediction. The code and pretrained models for reproducing our results will be released upon publication.
URL: https://openreview.net/forum?id=yoSD5BRFkV
---
Title: Neural Diversity Regularizes Hallucinations in Language Models
Abstract: Language models continue to hallucinate despite increases in parameters, compute, and data. We propose neural diversity — decorrelated parallel representations — as a principled mechanism that reduces hallucination rates at fixed parameter and data budgets. While existing mitigation strategies largely target accuracy, we provide the first formal tail bounds for hallucination probability in ensembled language models, reframing it as a second-moment reliability problem and explaining 96.2% of empirical reliability variation seen across parallel configurations. We introduce ND-LoRA (Neural Diversity Low-Rank Adaptation), combining parallel LoRA adapters with Barlow Twins regularization, and reduce hallucinations by up to 25.6% (and 14.6% on average) while preserving general accuracy. Ablations show LoRA adapters and regularization act synergistically, causal interventions prove neurodiversity as the mediating factor and correlational studies indicate scale: a 0.1% neural correlation increase is associated with a 3.8% hallucination increase. Finally, task-dependent optimality emerges: different tasks require different optimal amounts of neurodiversity. Together, our results highlight neural diversity as a third axis of scaling — orthogonal to parameters and data — to improve the reliability of language models at fixed budgets.
URL: https://openreview.net/forum?id=5l9ZflyApA
---
Title: IntervalGP-VAE: Learning Unobserved Confounders with Uncertainty for Personalized Causal Effect Estimation
Abstract: Estimating individual treatment effects (ITEs) in the presence of unobserved confounding remains a central challenge in causal inference. Existing proxy-based methods aim to recover latent confounders from observational proxies, but typically produce only point estimates without uncertainty quantification. This lack of uncertainty modeling provides incomplete and potentially insufficient information for downstream decision-making, especially when uncertainty is inherent in the data. We propose IntervalGP-VAE, a novel framework that combines variational autoencoders with Gaussian Processes (GPs) to model both the latent confounders and their associated uncertainty. At the core of our method is an interval-valued GP prior, which enables the model to capture a distribution over plausible latent confounders and treatment responses, rather than relying on potentially unreliable point estimates. This approach accounts for uncertainty arising from noisy and imperfect proxy variables and yields calibrated ITE intervals to support more robust causal decisions. We provide theoretical guarantees for identifiability of the latent confounder up to a smooth monotonic transformation under weak assumptions. Experiments on synthetic and semi-synthetic datasets demonstrate that IntervalGP-VAE achieves superior performance in ITE estimation and uncertainty calibration, outperforming existing methods.
URL: https://openreview.net/forum?id=vh7cshQCAE
---
Title: A quantitative analysis of semantic information in deep representations of text and images
Abstract: It was recently observed that the representations of different models that process identical or semantically related inputs tend to align. We analyze this phenomenon using the Information Imbalance, an asymmetric rank-based measure that quantifies the capability of a representation to predict another, providing a proxy of the cross-entropy which can be computed efficiently in high-dimensional spaces. By measuring the Information Imbalance between representations generated by DeepSeek-V3 processing translations, we find that semantic information is spread across many tokens, and that semantic predictability is strongest in a set of central layers of the network, robust across six language pairs. We measure clear information asymmetries: English representations are systematically more predictive than those of other languages, and DeepSeek-V3 representations are more predictive of those in a smaller model such as Llama3-8b than the opposite. In the visual domain, we observe that semantic information concentrates in middle layers for autoregressive models and in final layers for encoder models, and these same layers yield the strongest cross-modal predictability with textual representations of image captions. Notably, two independently trained models (DeepSeek-V3 and DinoV2) achieve stronger cross-modal predictability than the jointly trained CLIP model, suggesting that model scale may outweigh explicit multimodal training. Our results support the hypothesis of semantic convergence across languages, modalities, and architectures, while showing that directed predictability between representations varies strongly with layer-depth, model scale, and language.
URL: https://openreview.net/forum?id=sBnaFSIuGR
---
Title: Semantic-Aware Prefix Learning via Token Truncation for Efficient Image Generation
Abstract: Visual tokenizers play a central role in latent image generation by bridging high-dimensional images and tractable generative modeling.
However, most existing tokenizers are still trained with reconstruction-dominated objectives, which often yield latent representations that are only weakly grounded in high-level semantics. Recent approaches improve semantic alignment, but typically treat semantic signals as auxiliary regularization rather than making them functionally necessary for representation learning. We propose SMAP, a SeMantic-Aware Prefix tokenizer that injects semantic conditions as prefix-preserved invariants into a query-based 1D tokenization framework. To make semantics indispensable during training, SMAP introduces a tail token dropping strategy, which forces semantic conditions and early latent prefixes to bear increasing responsibility via progressive token truncation. This leads to information-ordered token sequences that support length-adaptive encoding and graceful truncation. To exploit the resulting latent space for generation, we further introduce CARD, a hybrid Causal AutoRegressive--Diffusion generator. CARD first models global structural dependencies autoregressively and then refines the conditional distribution via flow matching for high-fidelity synthesis. Extensive experiments on ImageNet show that SMAP consistently improves reconstruction quality across discrete and continuous tokenization settings, and that its semantically grounded latent space yields strong downstream generation performance with compact token budgets.
URL: https://openreview.net/forum?id=7257Oiv0hZ
---
Title: LLM-Guided Search for Deletion-Correcting Codes
Abstract: Finding deletion-correcting codes of maximum size has been an open problem for over 70 years, even for a single deletion. In this paper, we adapt FunSearch, a large language model (LLM)-guided evolutionary search, to discover functions that construct large deletion-correcting codes. For a single deletion, our search discovers a construction provably equivalent to the conjectured-optimal Varshamov-Tenengolts code. We study design choices for LLM-guided evolutionary search and find that, perhaps surprisingly, compute is better allocated to sampling more functions than extending reasoning per function or using larger models. We also find that evolving descriptions along with code is not beneficial, and propose deduplicating logically identical functions during evolution, which we find critical for search diversity. These results demonstrate the potential of LLM-guided evolutionary search for information theory and code design and represent the first application of such methods for constructing error-correcting codes.
URL: https://openreview.net/forum?id=qZ69Ozpo6v
---
Title: Learning to See the Unseen: Few-Shot 3D Scene Reconstruction via Diffusion and Gaussian Fields
Abstract: While recent methods have achieved impressive results in 3D reconstruction, they typically rely on dense multi-view inputs and often struggle with ambiguity in occluded or unobserved regions, particularly in complex scene layouts and background areas. We propose Learning to See the Unseen (LSU), a unified framework for high-fidelity 3D scene reconstruction from sparse or even single-image inputs by coupling generative novel-view synthesis with Gaussian-based scene reconstruction. Our approach introduces a Scene Diffusion Module (SDM) that conditions on sparse views and text prompts to synthesize consistent novel views. To improve spatial alignment across generated views, SDM incorporates a scene-level geometric supervision strategy that constrains the diffusion process using 3D structural consistency. Additionally, we design a geometry-aware Gaussian reconstruction module that leverages depth and surface normal priors to refine the reconstructed scene, improving geometric accuracy, background coherence, and rendering fidelity. Extensive experiments demonstrate that LSU achieves state-of-the-art performance on the RealEstate10K dataset and generalizes effectively to unseen domains, including KITTI and Mip-NeRF, recovering accurate global geometry while preserving fine-grained visual details across diverse scenes.
URL: https://openreview.net/forum?id=1jya5CBSwJ
---
Title: On the Role of Depth in Surgical Vision Foundation Models: An Empirical Study of RGB-D Pre-training
Abstract: Vision foundation models (VFMs) have emerged as powerful tools for surgical scene understanding. However, current approaches predominantly rely on unimodal RGB pre-training, overlooking the complex 3D geometry inherent to surgical environments. Although several architectures support multimodal or geometry-aware inputs in general computer vision, the benefits of incorporating depth information in surgical settings remain underexplored. We conduct a large-scale empirical study comparing eight ViT-based VFMs that differ in pre-training domain, learning objective, and input modality (RGB vs. RGB-D). For pre-training, we use a curated dataset of 1.4 million robotic surgical images paired with depth maps generated from an off-the-shelf network. We evaluate these models under both frozen-backbone and end-to-end fine-tuning protocols across eight surgical datasets spanning object detection, segmentation, depth estimation, and pose estimation. Our experiments yield several consistent findings. Models incorporating explicit geometric tokenization, such as MultiMAE, substantially outperform unimodal baselines across all tasks. Notably, geometric-aware pre-training enables remarkable data efficiency: models fine-tuned on just 25% of labeled data consistently surpass RGB-only models trained on the full dataset. Importantly, these gains require no architectural or runtime changes at inference; depth is used only during pre-training, making adoption straightforward. These findings suggest that multimodal pre-training offers a viable path towards building more capable surgical vision systems.
URL: https://openreview.net/forum?id=4UVdlpwcEc
---
Title: Unforgotten Safety: Preserving Safety Alignment of Large Language Models with Continual Learning
Abstract: The safety alignment of large language models (LLMs) is becoming increasingly important with their democratization. In this paper, we study the safety degradation that comes with adapting LLMs to new tasks. We attribute this safety compromise to catastrophic forgetting and frame the problem of preserving safety when fine-tuning as a continual learning (CL) problem. We consider the fine-tuning-as-a-service setup where users upload their data to a service provider to get a customized model that excels on the user’s selected task. We adapt several CL approaches from the literature and systematically evaluate their ability to mitigate safety degradation. These include regularization-based, memory-based, and model merging approaches. We consider two scenarios, (1) benign user data and (2) poisoned user data. Our results demonstrate that CL approaches consistently achieve lower attack success rates than standard fine-tuning. Among these, DER outperforms both other CL methods and existing safety-preserving baselines while maintaining task utility. These findings generalize across three downstream tasks (GSM8K, SST2, Code) and three model families (LLaMA2-7B, Mistral-7B, Gemma-2B), establishing CL as a practical solution to preserve safety.
URL: https://openreview.net/forum?id=Bm0fkQiPFq
---
Title: Layer-Wise Cognitive Specialization in Large Language Models: A Cross-Architecture Analysis of Concept Emergence
Abstract: This paper studies how internal representations change layer by layer in four language
models: DeepSeek-R1-Distill-Qwen-1.5B, Qwen3-4B-Thinking, Llama-3.1-8B-Instruct, and
Mistral-7B-Instruct-v0.2. We use 128 linear probes and activations from 215 questions across
16 cognitive categories to track when each category becomes easy to decode from model
states. We find three main results. First, the same broad ordering appears across models:
spatial navigation and logical reasoning become separable early, while pattern recognition
and executive function appear later. Second, most gains happen in the first third of layers
in all models, with clear differences in later layers. For example, Mistral-7B loses separability
in late layers (−1.4%), while Llama-8B shows the largest confidence increase (0.41-bit en
tropy reduction). Third, fresh paraphrase-based replication shows that late-layer category
decoding transfers across models (mean best accuracy 0.641 on 62 paraphrased prompts),
but exact emergence ordering does not replicate cleanly (mean rank correlation 0.016). We
validate results with bootstrap confidence intervals, confusion analysis, robust metrics, sig
nificance tests, paraphrase replication, intervention tests, and sanity controls. These findings
offer a practical map of where cognitive information appears and changes inside language
models while also clarifying which parts of that map are robust to prompt reformulation.
URL: https://openreview.net/forum?id=vtstJVjilF
---
Title: SSM-PixNav: State Space Models for Pixel-Guided Embodied Navigation
Abstract: While navigating a robot towards a specific object or image, how do we ensure it focuses on the correct location we are referring to? Object goal navigation, image goal navigation, goal instance navigation and pixel navigation are popular approaches to solve the problem. In this work, we focus on pixel navigation as it solves the problem more precisely by providing the agent with additional pixel-level guidance. Prior work has largely relied on RGB input; as a result, the policy lacks explicit geometric awareness, which can be important when visually similar regions differ in navigability. Additionally, recent work leverages transformer-based architectures to model temporal dependencies in observations, thereby increasing computational cost. Another practical limitation is the absence of an open benchmark dataset to reproduce the baselines. Through this work, we address these limitations along three directions. First, we introduce an RGBD-PixNav policy, a transformer-based architecture that incorporates depth directly into the policy. Second, to improve temporal modelling while maintaining computational efficiency, we employ Mamba, a recent State Space Model (SSM) architecture that enables lightweight sequential scanning of the observations. Building on this, we develop Causal SSM-based navigation policies and introduce a depth gating mechanism to regulate the contribution of depth features during policy learning. Third, to facilitate reproducible evaluation and future research, we curate the PixNav Trajectories dataset using HM3D scenes in Habitat-sim. Through extensive experiments, we establish an RGB-only baseline and extend to a transformer-based RGBD model and SSM-based variants. Results show that the proposed Causal SSM-RGB PixNav and Causal SSM-RGBD PixNav with depth gate consistently outperform other policy variants, improving the success rate by $\approx$0.4 while reducing model size to just half, $\approx$27M parameters. The models also demonstrate robustness to observation noise and varying history length. Code and dataset will be publicly released.
URL: https://openreview.net/forum?id=RmsMd5vdBf
---
Title: Anomaly-Gym: A Benchmark for Anomaly Detection in Reinforcement Learning Environments
Abstract: Anomaly detection (AD) is a key component for deploying reinforcement learning (RL) agents in safety-critical environments, enabling systems to identify unexpected conditions and trigger safe fallback behavior. Despite its importance, research on AD in RL settings is limited. Only a handful of methods have been proposed which - due to the absence of established evaluation scenarios - are evaluated on simple, small-scale, and self-proposed environments. This results in poor comparability and limits systematic analysis of the strengths and weaknesses of current approaches, thus fundamentally impeding progress in this direction. We address this problem by introducing Anomaly-Gym, a comprehensive evaluation suite for AD in RL settings. In contrast to prior work, Anomaly-Gym is based on principled design criteria that disentangle evaluation from methodology. By enforcing specific constraints on the environments and anomalies considered, we propose a broad spectrum of evaluation data that covers both simulated and real-world tasks. In total, our benchmark features 10 different environments, 25 anomaly types, 4 strength levels, as well as multiple sensor modalities. We demonstrate the importance of these different aspects in a series of experiments on pre-generated datasets. For instance, we show that simple methods, while generally neglected in previous work, achieve competitive scores for settings with observational disturbances. In contrast, detecting perturbations of actions or environment dynamics requires more complex methods. Our findings also highlight current challenges with anomaly detection on image data and provide directions for future research.
URL: https://openreview.net/forum?id=fDUb0A1DBA
---
Title: FARM: Enhancing Molecular Representations with Functional Group Awareness
Abstract: We introduce Functional Group-Aware Representations for Small Molecules (FARM), a novel foundation model designed to bridge the gap between SMILES (a linear string representation of molecular structures), natural language, and molecular graphs. The key innovation of FARM lies in its functional group (FG) annotation at the atomic level, which enables both FG-enhanced SMILES and FG graphs. SMILES are enriched with FG information to specify which functional group each atom belongs to, while the FG graph captures the molecular backbone by showing how the functional groups are connected. This tokenization not only injects chemical knowledge into SMILES but also expands the chemical lexicon, effectively bridging the gap between SMILES and natural language in terms of vocabulary size, making the sequences more suitable for Transformer-based models. FARM learns molecular representations from two complementary perspectives to encode both functional and structural information. Masked language modeling on FG-enhanced SMILES captures atom-level features enriched with FG context, while graph neural networks encode higher-level molecular topology by representing how functional groups are connected. Contrastive learning then aligns these two views into a unified embedding, ensuring that atom-level details and FG-level structure are jointly represented. This dual-level modeling is central to FARM’s ability to predict molecular properties accurately. We rigorously evaluate FARM on the MoleculeNet dataset, achieving state-of-the-art performance on 11 out of 13 tasks, and further validate its generalization on the photostability dataset for quantum mechanics properties. These results highlight FARM’s potential to improve molecular representation learning, demonstrate strong transfer learning capabilities across drug discovery and material design domains, and enable broad applications in pharmaceutical research and functional materials development. The code is available at: https://anonymous.4open.science/r/farm-E3CB
URL: https://openreview.net/forum?id=2All12TFlh
---
Title: MissingBench-Verified: Probing Vision-Language Models’ Inability to Detect Missing Object Parts
Abstract: Vision Language Models (VLMs) are well known for hallucinating non-existent objects in images. Objects with missing parts present a unique challenge for VLMs, stemming from both real-world knowledge bias and the scarcity of such images in training data. We present MissingBench-Verified, a benchmark designed to evaluate a specific and practically relevant scenario: when vision-language models fail to recognize that an essential component of an object has been removed. Across ten leading models, we observe consistent and significant failure rates that persist even when external tool evidence explicitly contradicts the model's visual perception. Error analysis reveals that models frequently dismiss tool outputs, attribute missing regions to occlusion or framing artifacts, or confabulate object attributes to reconcile the contradiction. We further ask whether granting models access to image processing tools (e.g., cropping, contrast adjustment) enables autonomous inspection to resolve these failures. We find that existing mitigation strategies, including tool-assisted verification, autonomous visual reasoning, longer reasoning durations, and fine-tuning on an easier dataset, provide negligible improvement, indicating that this failure mode cannot be addressed through current prompting or post-hoc correction techniques. Our findings highlight a fundamental limitation of current VLM for inspection and monitoring tasks and underscore the need for architectural or training-level interventions that enable models to override internal expectations when confronted with contradictory evidence.
URL: https://openreview.net/forum?id=OuqnHLrjB1
---
Title: Recursive Deep Inverse Reinforcement Learning
Abstract: Inferring an adversary’s goals from exhibited behavior is crucial for counterplanning and non-cooperative multi-agent systems in domains like cybersecurity, military, and strategy games. Deep Inverse Reinforcement Learning (IRL) methods based on maximum entropy principles show promise in recovering adversaries’ goals but are typically offline, require large batch sizes with gradient descent, and rely on first-order updates, limiting their applicability in real-time scenarios. We propose an online Recursive Deep Inverse Reinforcement Learning (RDIRL) approach to recover the cost function governing the adversary actions and goals. Specifically, we minimize an upper bound on the standard Guided Cost Learning (GCL) objective using sequential second-order Newton updates, akin to the Extended Kalman Filter (EKF), leading to a fast (in terms of convergence) learning algorithm. We demonstrate that RDIRL is able to recover cost and reward functions of expert agents in standard and adversarial benchmark tasks. Experiments on benchmark tasks show that our proposed approach outperforms several leading IRL algorithms.
URL: https://openreview.net/forum?id=bYItLUU135
---
Title: Not All Structure Is Learned: Disentangling Inherited and Learned Representations in Recurrent Networks
Abstract: Structure observed in trained recurrent networks may be inherited from input encodings rather than learned from data. We develop and apply a three-step decomposition to disentangle the two: (1) compare trained representations against untrained baselines to isolate input-driven structure, (2) compare against information-theoretic bounds to quantify what is achievable without learning, and (3) use causal interventions to test whether inherited and learned components are functionally used. Applied to GRUs trained via behavioral cloning on aliased navigation in a 127-node binary tree, the most prominent hidden-state feature, a depth gradient on PC1, is already present before training: an untrained GRU captures 96% of the trained correlation, reflecting input structure rather than learned spatial knowledge. What training adds is within-class node discrimination via sequential memory. Replacing depth-stratified observations with random class assignments eliminates the inherited axis; the GRU compensates with 7× greater learned spatial discrimination while maintaining comparable performance. PCA ablation reveals a double dissociation confirming that both inherited and learned components are causally involved in behavior. Applied to a non-hierarchical radial arm maze, the framework recovers an analogous inherited axis but qualitatively different learned structure: visit history tracking rather than spatial disambiguation.
URL: https://openreview.net/forum?id=1RfgHzf5IA
---
Title: Defending Membership Inference Attacks via Privacy-aware Sparsity Tuning
Abstract: Over-parameterized models are often vulnerable to membership inference attacks, which aim to determine whether a specific sample is included in the training set of a target. Previous weight regularization approaches typically impose uniform penalties on all parameters, leading to a suboptimal trade-off between model utility and privacy. In this work, we first show that only a small fraction of the parameters substantially impact privacy risk. Motivated by this analysis, we propose Privacy-aware Sparsity Tuning ($\textbf{PAST}$)—a novel privacy-preserving training method—by employing adaptive penalties to different model parameters. Our key idea behind PAST is to promote sparsity in model parameters that significantly contribute to privacy leakage risk. In particular, we construct the adaptive weight for each parameter based on its privacy sensitivity, i.e., the gradient of the loss gap with respect to the parameter. By using PAST, the network reduces the loss gap between members and non-members, thereby improving resistance to privacy attacks while preserving model utility. Extensive experiments consistently demonstrate the superiority of our defense method across various datasets, different network architectures, and diverse attack methods, achieving superior performance in the privacy-utility trade-off.
URL: https://openreview.net/forum?id=KlqlcnwqHw
---
Title: LLM-Guider: A Language-Guided Discovery of Symbolic Pruning Metrics for Post-Training Sparsity in LLMs
Abstract: Large Language Models (LLMs) have achieved remarkable advancements in natural language understanding, yet their mammoth size, coupled with substantial training and inference costs, can make them difficult to use in environments with limited resources. To address both memory and efficiency concerns, post-training unstructured sparsity techniques have emerged, focusing on developing optimal pruning criteria to eliminate redundant weights while maintaining performance. However, these approaches often rely on manually crafted pruning criteria, leading to sub-optimal solutions due to heuristic oversimplifications. Therefore, we introduce LLM-Guider, a language-guided symbolic formula optimization framework that seeks to discover optimal pruning criteria through a transparent and systematic process. LLM-Guider comprises three interrelated stages: example selection, formula generation, and formula evaluation, which collectively enable the efficient exploration of the formula space. In addition, LLM-Guider enables the incorporation of intuition, domain, and mathematical knowledge through role prompts, hints, and in-context examples. We also extend the standard set of aggregation strategies over a calibration dataset, resulting in never-seen-before pruning metrics.
Through extensive experiments, we demonstrate that formulas discovered through LLM-Guider is able to find formulas that outperform established baselines.
URL: https://openreview.net/forum?id=SlVQxEiYnY
---
Title: Object-Centric Representation Learning via Probabilistic Superpixel Coding
Abstract: Learning grounded object-centric representations that remain stable and invariant under
identity-preserving changes can improve robustness across diverse downstream visual tasks.
Most recent object-centric approaches are built on Slot Attention (SA), which learns object-
centric representations by assigning a set of (N) image features to a smaller set of (K) slots,
with each slot intended to represent a distinct object. Slot Attention typically initializes all
slots from a single shared distribution, making them largely exchangeable rather than spe-
cialized. As a result, the model struggles to learn dedicated slots that consistently bind to
particular object types and remain invariant under identity-preserving changes across scenes
or viewpoints. To address the binding of representations of an object with more permanent
(canonical) identifying characteristics of its type, we present Probabilistic Superpixel Coding
(PSC), which replaces slot specialization with an explicit identity codebook that tracks object
identity. Probabilistic Superpixel Coding factorizes each object into (i) an identity vector
zid selected from a shared codebook and (ii) a state vector zstate that captures instance-
specific variation. In this design, all occurrences of the same object type (e.g., orange) are
encouraged to retrieve the same identity entry, while zstate explains changes in pose, ap-
pearance, and context. We demonstrate the benefits of our method on multiple downstream
tasks, including scene generation, compositionality, and task adaptation, while remaining
competitive with slot-based baselines on widely used object discovery benchmarks
URL: https://openreview.net/forum?id=whbEYBdGGq
---
Title: Beyond NL2Code: A Systematic Survey of Multimodal Code Intelligence
Abstract: While Large Language Models (LLMs) have revolutionized text-to-code synthesis, conventional text-centric paradigms fail to capture the dense spatial hierarchies and structural constraints inherent in real-world visual contexts, such as user interfaces and scientific plots.
To bridge this gap, Multimodal Code Intelligence has emerged as a pivotal domain, empowering Vision-Language Models (VLMs) to translate visual perception into precise executable code. This paper presents a structured taxonomy of this rapidly evolving landscape, systematically categorizing the literature into four foundational domains: Graphical User Interfaces, Scientific Visualization, Structured Graphics, and Frontiers Frameworks. Within this framework, we systematically analyze tasks ranging from mainstream web and chart synthesis to complex emerging scenarios, such as programmatic visual manipulation and code-to-video generation. Through rigorous analysis of existing benchmarks and methodologies, we identify four pivotal technical shifts that may shape future research: the transition from imitation-based training to reward-driven optimization, the progression from static synthesis toward dynamic interaction, the evolution toward unified, general-purpose models, and the evolution from chat-based systems into autonomous agents. We envision this systematic survey as a foundational guide to accelerate future advancements in multimodal code intelligence. The trajectory of this field is rapidly shifting from merely extracting basic functional logic to synthesizing high-fidelity, aesthetically refined, and dynamically interactive outputs through iterative refinement. Ultimately, we posit that code constitutes the universal action space for multimodal general intelligence. By empowering AI systems to seamlessly translate complex visual intent into executable logic and autonomously navigate digital environments, visually-grounded code generation marks a definitive breakthrough toward autonomous software agents. An ongoing, dynamically updated project and resources associated with this survey have been released at
\href{https://anonymous.4open.science/r/Awesome-Multimodal-LLM-for-Code-2031}{anonymous repo}.
URL: https://openreview.net/forum?id=pn2sdu3Vrf
---
Title: Behavioral Inference at Scale: The Fundamental Asymmetry Between Motivations and Belief Systems
Abstract: We establish empirical bounds on behavioral inference through controlled experiments at scale: LLM-based agents assigned one of 36 behavioral profiles (9 belief systems x 4 motivations) generate over 1.5 million behavioral sequences across 17,411 games in grid-world environments, providing ground truth unavailable in human behavioral studies. After model-specific filtering, BiLSTM classification experiments use 344,365 sequences from 4,064 games; Longformer experiments use 267,063 sequences from 3,531 games. Rather than asking whether inference has limits, we ask how large those limits are, where they concentrate, and why. A fundamental asymmetry emerges in both magnitude and structure. Motivations achieve 98-100% inference accuracy and recover 97% of available mutual information across all architectures. Belief systems plateau at 24% for LSTMs regardless of capacity, recovering only 30% of available information, a 3.3x asymmetry in information extraction efficiency. Transformer architectures with 9-stage curriculum learning reach 49% alignment accuracy, doubling LSTM performance and demonstrating that the recurrent ceiling is architectural rather than fundamental.
Yet even this improvement leaves belief systems correctly classified less than half the time, with per-alignment accuracy ranging from 1% (True~Neutral) to 72% (Lawful~Evil). Confusion analysis maps the failure structure precisely: a "neutral zone" of behavioral ambiguity extends beyond True~Neutral to encompass Good alignments, where prosocial behavior is indistinguishable from rule-following or balance-keeping. Combined motivation and belief inference yields 17.6x improvement over random baseline for full 36-class profile classification, while establishing that the bottleneck is entirely located in belief system inference.
Signal enhancement and explanatory queries yield only marginal LSTM gains (+3.8%), confirming that the ceiling is information-theoretic rather than data-limited. These bounds have direct implications for any system relying on behavioral monitoring to infer agent values.
URL: https://openreview.net/forum?id=aDMDqtw63H
---
Title: Revisiting Neighbourhoods in Mean Field Reinforcement Learning
Abstract: Many multi-agent reinforcement learning (MARL) algorithms do not scale well as the number of agents increases due to an exponential time and space complexity dependency on the number of agents in the environment. Mean field theory has been used to address this problem by approximating the effect of neighbourhoods of agents by a single representative agent. While this approximation allows MARL algorithms to scale to environments with many agents, approaches typically assumed that agents 1) inside a neighbourhood are homogeneous, and 2) outside a neighbourhood have no influence (and can therefore be ignored). This paper relaxes these assumptions and proposes a novel framework, mean field attention (MFA), which uses an attention mechanism for local responses and the mean field approximation for global responses. We implement MFA with two new algorithms leveraging Q-learning and actor-critic. These novel MFA algorithms consistently outperform other MARL algorithms, including prior mean field-based algorithms, across multiple metrics and benchmarks.
URL: https://openreview.net/forum?id=PQ5R7K0WDc
---
Title: The Perplexity Paradox: Why Code Compresses Better Than Math in LLM Prompts
Abstract: In "Compress or Route?" (Anonymous, 2026), we found that code generation and chain-of-thought reasoning respond differently to prompt compression: code tolerates aggressive compression (r >= 0.6) while reasoning degrades gradually. That initial study, however, was limited to a single code benchmark (HumanEval, 164 problems), left the proposed "perplexity paradox" mechanism unvalidated, and provided no adaptive algorithm. This paper addresses all three gaps. First, we validate the task-dependent compression hypothesis across six code benchmarks (HumanEval, MBPP, HumanEval+, MultiPL-E in Python/JavaScript/Java) and four reasoning benchmarks (GSM8K, MATH, ARC-Challenge, MMLU-STEM), demonstrating that the compression threshold (r >= 0.6) generalizes across programming languages and problem difficulties. Second, we conduct the first per-token perplexity analysis of compression decisions (n = 723 tokens), revealing a "perplexity paradox": code syntax tokens (which appear unusual to language models) are preserved, while numerical values in math problems (which follow predictable syntactic patterns) are pruned despite being task-critical. In a controlled signature preservation experiment, we demonstrate a +34 percentage point recovery in pass rate (5.3% baseline -> 39.3% with signature injection; Cohen's h = 0.890, very large effect), with NameError rates dropping from 86.1% to 6.1%. Third, we propose TAAC (Task-Aware Adaptive Compression), a quality-gated algorithm that dynamically adjusts compression based on predicted quality degradation, achieving 22% cost reduction with 96% quality preservation, outperforming fixed-ratio compression by 7%. Our MBPP validation (n = 1,800 trials across 6 compression ratios) confirms compression tolerance varies systematically with ratio: 3.6% at r = 0.3, 11.3% at r = 0.4, 23.3% at r = 0.5, 32.3% at r = 0.6, 42.6% at r = 0.7, and 54.6% at r = 1.0 (uncompressed baseline). Code and data are included in anonymized supplementary material for double-blind review.
URL: https://openreview.net/forum?id=H0zAqyli0n
---
Title: Hyperparameter Optimization for Small-Sample Machine Learning via Design of Experiments
Abstract: While current machine learning hyperparameter tuning methods have been thoroughly tested and show consistently high performance in large datasets, few studies have made efforts to rigorously assess their performance in small data regimes. Studies that have examined hyperparameter optimization in small datasets have found reduced generalization performance, poor correlation between validation and test error, and overly optimistic error estimates associated with the chosen hyperparameter. This has been observed across different hyperparameter optimization methods, including grid search and Bayesian optimization. We implement design of experiments principles to mitigate the bias between validation error and generalizable test error when hyperparameter tuning. Specifically, we utilize a surface fitted to a space-filling design on the hyperparameter space to generate optimal hyperparameter sets. Using fourteen publicly available datasets and repeated experiments via Monte Carlo simulation, we show that this method has similar generalizable test error compared to both grid search and Bayesian Optimization, but the bias between the validation error and generalizable test error is drastically reduced by 80-96\% compared to both methods. As a secondary outcome of this work, we find that none of the evaluated methods of hyperparameter optimization offer consistent improvement over untuned models in our experiments, raising questions about the general efficacy of hyperparameter optimization in small sample regimes.
URL: https://openreview.net/forum?id=bf03USBnfe
---
Title: Leveraging Perturbation Sensitivity for Hallucination Detection in Large Language Models
Abstract: Hallucination detection is essential for ensuring the reliability of large language models. Internal representation–based methods have emerged as the prevailing direction for detecting hallucinations, yet the internal representations often fail to yield clear separability between truthful and hallucinatory content. To address this challenge, we study the separability of the sensitivity to prompt-induced perturbations in the internal representations. A theory is established to show that, with non-negligible probability, each sample admits a prompt under which truthful samples exhibit greater sensitivity to prompt-induced perturbations than hallucinatory samples. When the theory is applied to the representative datasets, the probability reaches nearly 99%, suggesting that sensitivity to perturbations provides a discriminative indicator. Building on this insight, we propose Sample-Specific Prompting (SSP), which adaptively selects prompts to perturb the model’s internal states and measures the resulting sensitivity as a detection indicator. Extensive experiments across multiple benchmarks demonstrate that SSP consistently outperforms existing hallucination detection methods, validating the practical effectiveness of our method SSP in hallucination detection.
URL: https://openreview.net/forum?id=cA8ItlIeY5
---
Title: Discovering Hidden Algebraic Structures via Transformers with Rank-Aware Beam GRPO
Abstract: Recent efforts have extended the capabilities of transformers in logical reasoning and symbolic computations. In this work, we investigate their capacity for non-linear latent pattern discovery in the context of functional decomposition, focusing on the challenging algebraic task of multivariate polynomial decomposition. This problem, with widespread applications in science and engineering, is proved to be NP-hard, and demands both precision and insight. Our contributions are threefold: First, we develop a synthetic data generation pipeline providing fine-grained control over problem complexity. Second, we train transformer models via supervised learning and evaluate them across four key dimensions involving scaling behavior and generalizability. Third, we propose Beam Grouped Relative Policy Optimization (BGRPO), a rank-aware reinforcement learning method suitable for hard algebraic problems. Finetuning with BGRPO improves accuracy while reducing beam width by up to half, resulting in approximately 75% lower inference compute. Additionally, our model demonstrates competitive performance in polynomial simplification, outperforming Mathematica in various cases.
URL: https://openreview.net/forum?id=Vxf8QDIA6Z
---
Title: Dynamic Subspace Estimation from Undersampled Data using Grassmannian Geodesics
Abstract: This work considers recovering a sequence of low-rank matrices from undersampled measurements, where the underlying subspace varies across samples over time. Existing works involve concatenating all of the samples from each time point to recover the underlying matrix
under the assumption that the data are well-approximated by a single, static subspace. However, this assumption is inappropriate for applications where the best low-rank approximations vary over time. To address this issue, we propose a Riemannian block majorize minimization algorithm that constrains the time-varying subspaces as a geodesic along the Grassmann manifold. Our proposed method can faithfully estimate the best-fit subspaces at each time point, even when there are fewer samples at each time point than the subspace dimension. Theoretically, we show that our algorithm enjoys a monotonically non-increasing objective function while converging to an $\epsilon$-stationary point within $\widetilde{\mathcal{O}}(\epsilon^{-2})$ iterations. We demonstrate the effectiveness of our algorithm on synthetic, dynamic fMRI, and video data, where the samples at each time are either compressed or partially missing.
URL: https://openreview.net/forum?id=W0ZNwXoBeh
---
Title: Entropy-Guided Sampling of Flat Modes in Discrete Spaces
Abstract: Sampling from flat modes in discrete spaces is a crucial yet under-explored direction. Flat modes represent robust solutions and have broad applications in combinatorial optimization and discrete generative modeling. However, existing sampling algorithms often overlook the mode volume and struggle to capture flat modes effectively. To address this limitation, we propose Entropic Discrete Langevin Proposal (EDLP), which incorporates local entropy into the sampling process through a continuous auxiliary variable under a joint distribution. The
local entropy term guides the discrete sampler toward flat modes with a small overhead. We provide non-asymptotic convergence guarantees for EDLP in locally log-concave discrete distributions. Empirically, our method consistently outperforms traditional approaches across tasks that require sampling from flat basins, including Bernoulli distributions, restricted Boltzmann machines, combinatorial optimization, and binary neural networks.
URL: https://openreview.net/forum?id=42MHswkKCW
---
Title: Learning with Local Search MCMC Layers
Abstract: Integrating combinatorial optimization layers into neural networks has recently attracted significant research interest. However, many existing approaches lack theoretical guarantees or fail to perform adequately when relying on inexact solvers. This is a critical limitation, as many operations research problems are NP-hard, often necessitating the use of neighborhood-based local search heuristics. In this paper, we introduce a theoretically-principled approach for learning with such inexact solvers. Inspired by the connection between simulated annealing and Metropolis-Hastings, we propose to transform problem-specific neighborhood systems used in local search heuristics into proposal distributions, implementing MCMC on the set of feasible solutions. This allows us to construct differentiable, stochastic combinatorial layers and associated loss functions. Replacing an exact solver by a local search strongly reduces the computational burden of learning on many applications. We demonstrate our approach on a dynamic vehicle routing problem with time windows, binary vector and k-subset prediction tasks, as well as a multi-dimensional knapsack decision-focused learning problem.
URL: https://openreview.net/forum?id=M2a0mbACNf
---