Hello folks,
Continuous diffusion dominates image generation. LLMs process text through continuous embeddings. So why does discrete diffusion still win for language?
CANDI explains why — it’s a “temporal dissonance”: at large vocabulary sizes, Gaussian noise destroys token identity way before it meaningfully degrades the continuous signal. The model can either learn discrete conditional structure or continuous geometry, but not both simultaneously.
The fix? Keep some tokens clean as anchors for discrete structure, corrupt the rest with Gaussian noise. Decoupling the two lets the model learn both simultaneously — enabling off-the-shelf classifier guidance and better low-NFE generation.
This Monday, Patrick Pynadath will present his paper CANDI: Hybrid Discrete-Continuous Diffusion Models.
Title: CANDI: Hybrid Discrete-Continuous Diffusion Models
Meeting Link: click here
Time: Mar 9 (Monday) 1pm ET / 10am PT / 6pm CET / 10:30pm IST
Paper: https://arxiv.org/abs/2510.22510
Prior knowledge:
Fundamentals of discrete diffusion (video by Sasha Rush)
Abstract: While continuous diffusion has shown remarkable success in continuous domains such as image generation, its direct application to discrete data has underperformed compared to purely discrete formulations. This gap is counterintuitive, given that continuous diffusion learns score functions that enable joint evolution across multiple positions. To understand this gap, we introduce token identifiability as an analytical framework for understanding how Gaussian noise corrupts discrete data through two mechanisms: discrete identity corruption and continuous rank degradation. We reveal that these mechanisms scale differently with vocabulary size, creating a temporal dissonance: at noise levels where discrete corruption preserves enough structure for conditional learning, continuous denoising is trivial; at noise levels where continuous denoising is meaningful, discrete corruption destroys nearly all conditional structure. To solve this, we propose CANDI (Continuous ANd DIscrete diffusion), a hybrid framework that decouples discrete and continuous corruption, enabling simultaneous learning of both conditional structure and continuous geometry. We empirically validate the temporal dissonance phenomenon and demonstrate that CANDI successfully avoids it. This unlocks the benefits of continuous diffusion for discrete spaces: on controlled generation, CANDI enables classifier-based guidance with off-the-shelf classifiers through simple gradient addition; on text generation, CANDI outperforms masked diffusion at low NFE, demonstrating the value of learning continuous gradients for discrete spaces. We include the code on the project page available here: https://patrickpynadath1.github.io/candi-lander/. 
