Daily TMLR digest for Nov 27, 2025
TMLR
New certifications
==================
J2C Certification: Adaptive Mesh Quantization for Neural PDE Solvers
Winfried van den Dool, Maksim Zhdanov, Yuki M Asano, Max Welling
https://openreview.net/forum?id=NN17y897WG
---
Accepted papers
===============
Title: Adaptive Mesh Quantization for Neural PDE Solvers
Authors: Winfried van den Dool, Maksim Zhdanov, Yuki M Asano, Max Welling
Abstract: Physical systems commonly exhibit spatially varying complexity, presenting a significant challenge for neural PDE solvers. While Graph Neural Networks can handle the irregular meshes required for complex geometries and boundary conditions, they still apply uniform computational effort across all nodes regardless of the underlying physics complexity. This leads to inefficient resource allocation where computationally simple regions receive the same treatment as complex phenomena. We address this challenge by introducing Adaptive Mesh Quantization: spatially adaptive quantization across mesh node, edge and cluster features, dynamically adjusting the bit-width used by a quantized model.
We propose an adaptive bit-width allocation strategy driven by a lightweight auxiliary model that identifies high-loss regions in the input mesh. This enables dynamic resource distribution in the main model, where regions of higher difficulty are allocated increased bit-width, optimizing computational resource utilization. We demonstrate our framework's effectiveness by integrating it with two state-of-the-art models, MP-PDE and GraphViT, to evaluate performance across multiple tasks: 2D Darcy flow, large-scale unsteady fluid dynamics in 2D, steady-state Navier–Stokes simulations in 3D, and a 2D hyper-elasticity problem. Our framework demonstrates consistent Pareto improvements over uniformly quantized baselines, yielding up to 50\% improvements in performance at the same cost.
URL: https://openreview.net/forum?id=NN17y897WG
---
Title: Image and Video Quality Assessment using Prompt-Guided Latent Diffusion Models for Cross-Dataset Generalization
Authors: Shankhanil Mitra, Diptanu De, Shika Rao, Rajiv Soundararajan
Abstract: The design of image and video quality assessment (QA) algorithms is extremely important
to benchmark and calibrate user experience in modern visual systems. A major drawback
of the state-of-the-art QA methods is their limited ability to generalize across diverse image
and video datasets with reasonable distribution shifts. In this work, we leverage the
denoising process of diffusion models for generalized image QA (IQA) and video QA (VQA)
by understanding the degree of alignment between learnable quality-aware text prompts
and images or video frames. In particular, we learn cross-attention maps from intermediate
layers of the denoiser of latent diffusion models (LDMs) to capture quality-aware representations
of images or video frames. Since applying text-to-image LDMs for every video frame
is computationally expensive for videos, we only estimate the quality of a frame-rate subsampled
version of the original video. To compensate for the loss in motion information due
to frame-rate sub-sampling, we propose a novel temporal quality modulator. Our extensive
cross-database experiments across various user-generated, synthetic, low-light, frame-rate
variation, ultra high definition, and streaming content-based databases show that our model
can achieve superior generalization in both IQA and VQA.
URL: https://openreview.net/forum?id=FjhvVevAoQ
---
Title: Visual-Word Tokenizer: Beyond Fixed Sets of Tokens in Vision Transformers
Authors: Leonidas Gee, Wing Yan Li, Viktoriia Sharmanska, Novi Quadrianto
Abstract: The cost of deploying vision transformers increasingly represents a barrier to wider industrial adoption. Existing compression techniques require additional end-to-end fine-tuning or incur a significant drawback to energy efficiency, making them ill-suited for online (real-time) inference, where a prediction is made on any new input as it comes in. We introduce the Visual-Word Tokenizer (VWT), a training-free method for reducing energy costs while retaining performance. The VWT groups visual subwords (image patches) that are frequently used into visual words, while infrequent ones remain intact. To do so, intra-image or inter-image statistics are leveraged to identify similar visual concepts for sequence compression. Experimentally, we demonstrate a reduction in energy consumed of up to 47%. Comparative approaches of 8-bit quantization and token merging can lead to significantly increased energy costs (up to 500% or more). Our results indicate that VWTs are well-suited for efficient online inference with a marginal compromise on performance. The experimental code for our paper is also made publicly available.
URL: https://openreview.net/forum?id=YYOS1FHYG3
---
Title: Enhancing Physics-Informed Neural Networks Through Feature Engineering
Authors: Shaghayegh Fazliani, Zachary Frangella, Madeleine Udell
Abstract: Physics-Informed Neural Networks (PINNs) seek to solve partial differential equations (PDEs) with deep learning. Mainstream approaches that deploy fully-connected multi-layer deep learning architectures require prolonged training to achieve moderate accuracy, while recent work on feature engineering allows higher accuracy and faster convergence. This paper introduces SAFE-NET, a Single-layered Adaptive Feature Engineering NETwork that improves errors with far fewer parameters than baseline feature engineering methods. SAFE-NET returns to basic ideas in machine learning, using Fourier features,
a simplified single hidden layer network architecture, and an effective optimizer that improves the conditioning of the PINN optimization problem. Numerical results show that SAFE-NET converges faster and typically outperforms deeper networks and more complex architectures. It consistently uses fewer parameters --- on average, 53% fewer than the competing feature engineering methods and 70-100$\boldsymbol{\times}$ fewer than state-of-the-art large-scale architectures --- while achieving comparable accuracy in less than 30% of the training epochs. Moreover, each SAFE-NET epoch is 95% faster than those of competing feature engineering approaches. These findings challenge the prevailing belief that modern PINNs effectively learn relevant features and highlight the efficiency gains possible through feature engineering.
URL: https://openreview.net/forum?id=J25OqR4pBB
---
Title: SaFARi: State-Space Models for Frame-Agnostic Representation
Authors: Hossein Babaei, Mel White, Sina Alemohammad, Richard Baraniuk
Abstract: State-Space Models (SSMs) have re-emerged as a powerful tool for online function approximation, and as the backbone of machine learning models for long-range dependent data.
However, to date, only a few polynomial bases have been explored for this purpose, and the state-of-the-art implementations were built upon the best of a few limited options. In this paper, we present a generalized method for building an SSM with any frame or basis, rather than being restricted to polynomials.
This framework encompasses the approach known as HiPPO, but also permits an infinite diversity of other possible "species" within the SSM architecture. We dub this approach SaFARi: SSMs for Frame-Agnostic Representation.
URL: https://openreview.net/forum?id=UAgxU8gBtv
---
Title: MaskRIS: Semantic Distortion-aware Data Augmentation for Referring Image Segmentation
Authors: Minhyun Lee, Seungho Lee, Song Park, Dongyoon Han, Byeongho Heo, Hyunjung Shim
Abstract: Referring Image Segmentation (RIS) is an advanced vision-language task that involves identifying and segmenting objects within an image as described by free-form text descriptions. While previous studies focused on aligning visual and language features, exploring training techniques, such as data augmentation, remains underexplored. In this work, we explore effective data augmentation for RIS and propose a novel training framework called Masked Referring Image Segmentation (MaskRIS). We observe that the conventional image augmentations fall short of RIS, leading to performance degradation, while simple random masking significantly enhances the performance of RIS. MaskRIS uses both image and text masking, followed by Distortion-aware Contextual Learning (DCL) to fully exploit the benefits of the masking strategy. This approach can improve the model's robustness to occlusions, incomplete information, and various linguistic complexities, resulting in a significant performance improvement. Experiments demonstrate that MaskRIS can easily be applied to various RIS models, outperforming existing methods in both fully supervised and weakly supervised settings. Finally, MaskRIS achieves new state-of-the-art performance on RefCOCO, RefCOCO+, and RefCOCOg datasets. Code is available at https://github.com/naver-ai/maskris.
URL: https://openreview.net/forum?id=EtK4madHmc
---
Title: Rewarding the Rare: Maverick-Aware Shapley Valuation in Federated Learning
Authors: Mengwei Yang, Baturalp Buyukates, Athina Markopoulou
Abstract: Federated Learning (FL) allows clients to train a model collaboratively without sharing their private data. Shapley value (SV) provides a principled way to quantify client contributions in FL. However, existing SV methods use uniform per-class weighting during validation, treating all classes as equally important. This uniform weighting breaks down in the presence of clients with underrepresented or rare classes, also referred to as Mavericks. Such clients are often undervalued due to lower model performance on these challenging classes, despite their critical role in improving generalization. To address this, we introduce a Maverick-aware Shapley valuation framework that reweights validation scores based on per-class accuracy, assigning greater importance to classes where models perform poorly. Building on this, we design FedMS, a Maverick-Shapley client selection mechanism that leverages our refined contribution scores to guide intelligent client selection. Experiments on benchmark datasets demonstrate that FedMS improves model performance and better recognizes valuable client contributions, even under scenarios involving adversaries, free-riders, and skewed or rare-class distributions.
URL: https://openreview.net/forum?id=JtybGfTUdq
---
Title: Tree Structure for the Categorical Wasserstein Weisfeiler-Lehman Graph Kernel
Authors: Keishi Sando, Tam Le, Hideitsu Hino
Abstract: The Wasserstein Weisfeiler-Lehman~(WWL) graph kernel is a popular and efficient approach, utilized in various kernel-dependent machine learning frameworks for practical applications with graph data. It incorporates optimal transport geometry into the Weisfeiler-Lehman graph kernel, to mitigate the information loss inherent in aggregation strategies of graph kernels. While the WWL graph kernel demonstrates superior performance in many applications, it suffers a drawback in its computational complexity, i.e., at least $\mathcal{O}(n_{1} n_{2})$, where $n_{1}, n_{2}$ denote the number of vertices in the input graphs. Consequently, it hinders the practical applicability of the WWL graph kernel, especially in large-scale settings. In this paper, we propose the \emph{Tree Wasserstein Weisfeiler-Lehman}~(TWWL) algorithm, which leverages a \emph{tree structure} to scale up the exact computation of the WWL graph kernel for graph data with categorical node labels. In particular, the computational complexity of the TWWL algorithm is $\mathcal{O}(n_{1} + n_{2})$, which enables its application to large-scale graphs. Numerical experiments demonstrate that the performance of the proposed algorithm compares favorably with baseline kernels, while its computation is several orders of magnitude faster than the classic WWL graph kernel. This paves the way for applications in large-scale datasets where the WWL kernel is computationally prohibitive.
URL: https://openreview.net/forum?id=VwoSsFK22P
---
New submissions
===============
Title: Statistical Inference for Generative Model Comparison
Abstract: Generative models have achieved remarkable success across a range of applications, yet their evaluation still lacks principled uncertainty quantification. In this paper, we develop a method for comparing how close different generative models are to the underlying distribution of test samples. Particularly, our approach employs the Kullback-Leibler (KL) divergence to measure the distance between a generative model and the unknown test distribution, as KL requires no tuning parameters such as the kernels used by RKHS-based distances. And the relative KL divergence is the only $f$-divergence that admits a crucial cancellation of the hard-to-estimate term to enable the faithful uncertainty quantification. Furthermore, we extend our method to comparing conditional generative models and leverage Edgeworth expansions to address limited-data settings. On simulated datasets with known ground truth, we show that our approach realizes effective coverage rates, and has higher power compared to kernel-based methods. When applied to generative models on image and text datasets, our procedure yields conclusions consistent with benchmark metrics but with statistical confidence.
URL: https://openreview.net/forum?id=PXL6SBxh0q
---
Title: Combinatorial Capacity of modReLU Complex Networks: VC-Dimension Bounds and Lower Limits
Abstract: Complex-valued neural networks (CVNNs) are increasingly used in settings where both
magnitude and phase of the signal carry information. In particular, deep networks with
the modReLU activation function have become standard in applications such as MRI
reconstruction, radar, and complex-valued time-series modeling. While approximation
properties of such networks have recently been analyzed in detail, their statistical
capacity in the sense of VC-dimension has not, to the best of our knowledge, been studied.
In this paper we formalize a natural class of fully connected deep complex-valued networks
with modReLU activation and real sign output, and view them as binary classifiers on
$\mathbb{R}^{2d}$ via the usual realification. Using tools from real algebraic geometry and a
VC-dimension bound for semi-algebraic concept classes due to Goldberg and Jerrum,
together with quantitative bounds for quantifier elimination, we prove that for any
architecture with $W$ real parameters and depth $L$, the VC-dimension of the corresponding
hypothesis class is at most on the order of $W^2 \log W$, with a universal constant
independent of the particular architecture.
On the other hand, by restricting to real inputs and parameters and exploiting results of
Harvey, Liaw, and Mehrabian and of Bartlett et al. on deep networks with piecewise-linear
activations, we obtain lower bounds of order $WL \log(W/L)$ for suitable depth-$L$
architectures within the modReLU class. Thus the VC-dimension of these networks grows
at least linearly in both $W$ and $L$, and at most quadratically in $W$ up to a logarithmic
factor. Closing this gap is an interesting open problem.
URL: https://openreview.net/forum?id=jfeJnfST36
---
Title: PruneFuse: Efficient Data Selection via Weight Pruning and Network Fusion
Abstract: Efficient data selection is crucial for enhancing the training efficiency of deep neural networks and minimizing annotation requirements. Traditional methods often face high computational costs, limiting their scalability and practical use. We introduce PruneFuse, a novel strategy that leverages pruned networks for data selection and later fuses them with the original network to optimize training.
PruneFuse operates in two stages: First, it applies structured pruning to create a smaller pruned network that, due to its structural coherence with the original network, is well-suited for the data selection task. This small network is then trained and selects the most informative samples from the dataset. Second, the trained pruned network is seamlessly fused with the original network. This integration leverages the insights gained during the training of the pruned network to facilitate the learning process of the fused network while leaving room for the network to discover more robust solutions. Extensive experimentation on various datasets demonstrates that PruneFuse significantly reduces computational costs for data selection, achieves better performance than baselines, and accelerates the overall training process.
URL: https://openreview.net/forum?id=BvnxenZwqY
---
Title: Incremental Feature Selection in Dynamic Incomplete Ordered Decision Systems
Abstract: Incremental feature selection aims to efficiently identify key features from dynamic data. However, existing feature selection algorithms for dynamic incomplete ordered data often rely on upper and lower approximations while overlooking the impact of inter-feature relationships across different decision classes. This can lead to reduced computational efficiency and suboptimal classification accuracy. To address these issues, this paper proposes an incremental feature selection method based on expanded dominance matrices for incomplete ordered decision systems. Firstly, we propose to use non-dominant relationships between classes as a measure of attribute importance, thereby avoiding the computational complexity of traditional lower and upper approximation. Furthermore, to maintain efficiency and accuracy in dynamic data environments which involve frequent object addition and deletion, we propose two matrix-based incremental update mechanisms: matrix-based non-dominance attribute reduction for addition (MNAR-A) and matrix-based non-dominance attribute reduction for deletion (MNAR-D). These mechanisms are crucial for efficiently updating the feature subset when new objects are added or existing objects are removed, ensuring the algorithm remains effective and avoids recomputing from scratch. Experimental results on the UCI dataset showed that the proposed algorithm achieved a 1.3\(\times\) speedup and delivered a 7\% relative accuracy gain compared to the state-of-the-art method on average.
URL: https://openreview.net/forum?id=S4cOJfWCZZ
---
Title: Rethinking Developmental Curricula for Contrastive Visual Learning
Abstract: While large machine learning models have achieved remarkable results, they still fall short of the efficiency and adaptability characteristic of human perception. Inspired by infant visual development, we explore developmental curriculum learning strategies for contrastive learning, systematically isolating their effects under controlled conditions. Within a virtual environment, we modulated four dynamic factors, namely image blur, lighting complexity, avatar movement speed, and image complexity, to simulate developmental progression. However, none of these conditions improved downstream classification performance compared with a stable train setting. We then repeated the experiments on the real-world SAYCam dataset using dynamic movement speed and image complexity separately and obtained consistent results. These findings suggest that performance gains attributed to developmental learning do not arise directly from commonly assumed perceptual factors, which challenges the assumption that developmental-like progression inherently benefits learning and highlights the need for more principled curriculum design mechanisms. Our results offer a new perspective on curriculum design for self-supervised learning.
URL: https://openreview.net/forum?id=MZ6eV86G7G
---
Title: Approximately Equivariant Recurrent Generative Models for Quasi-Periodic Time Series with a Progressive Training Scheme
Abstract: We present a simple yet effective generative model for time series, based on a Recurrent Variational Autoencoder that we refer to as RVAE-ST. Recurrent layers often struggle with unstable optimization and poor convergence when modeling long sequences. To address
these limitations, we introduce a progressive training scheme that gradually increases the sequence length, stabilizing optimization and enabling consistent learning over extended horizons. By composing known components into a recurrent, approximately time-shift-equivariant topology, our model introduces an inductive bias that aligns with the structure of quasi-periodic and nearly stationary time series. Across several benchmark datasets, RVAE-ST matches or surpasses state-of-the-art generative models, particularly on quasi-periodic data, while remaining competitive on more irregular signals. Performance is evaluated through ELBO, Fréchet Distance, discriminative metrics, and visualizations of the learned latent embeddings.
URL: https://openreview.net/forum?id=KHk5EECG3Z
---
Title: SURFACEBENCH: Can Self-Evolving LLMs Find the Equations of 3D Scientific Surfaces?
Abstract: Equation discovery from data is a core challenge in machine learning for science, requiring the recovery of concise symbolic expressions that govern complex physical and geometric phenomena. Recent approaches with large language models (LLMs) show promise in symbolic regression, but their success often hinges on memorized formulas or overly simplified functional forms. Existing benchmarks exacerbate this limitation: they focus on scalar functions, ignore domain grounding, and rely on brittle string-matching based metrics that fail to capture scientific equivalence. We introduce SurfaceBench, the first comprehensive benchmark for symbolic surface discovery. SurfaceBench comprises 183 tasks across 15 categories of symbolic complexity, spanning explicit, implicit, and parametric equation representation forms. Each task includes ground-truth equations, variable semantics, and synthetically sampled three dimensional data. Unlike prior SR datasets, our tasks reflect surface-level structure, resist LLM memorization through novel symbolic compositions, and are grounded in scientific domains such as fluid dynamics, robotics, electromagnetics, and geometry. To evaluate equation discovery quality, we pair symbolic checks with geometry-aware metrics such as Chamfer and Hausdorff distances, capturing both algebraic fidelity and spatial reconstruction accuracy. Our experiments reveal that state-of-the-art frameworks, while occasionally successful on specific families, struggle to generalize across representation types and surface complexities. SurfaceBench thus establishes a challenging and diagnostic testbed that bridges symbolic reasoning with geometric reconstruction, enabling principled benchmarking of progress in compositional generalization, data-driven scientific induction, and geometry-aware reasoning with LLMs.
URL: https://openreview.net/forum?id=sHLTzkczSi
---