Session 25: FLARE: Diffusion for Hybrid Language Model
Diffusion LLM
Hello folks,
Sequential decoding creates a latency bottleneck for AR LLMs. While combining hybrid attention with parallel diffusion models (dLLMs) could speed this up, doing so is difficult due to a number of reasons:
AR-to-dLLM conversion often doesn’t preserve AR quality
Hybrid attention and masking makes training and serving difficult
To address these issues, the authors propose FLARE, a conversion framework for hybrid-attention LLMs. They identify that transfer data quality—not loss formulation—is the key to preserving model capabilities during conversion.
FLARE unifies AR and diffusion training objectives while also using hardware-aware kernels. This allows a single checkpoint to seamlessly support both AR-style verified decoding and diffusion-style parallel denoising.
FLARE rivals top open-source dLLMs across scales and consistently boosts throughput in single-GPU serving. The study concludes that practical dLLM deployment requires the joint design of data, training objectives, architectures, and inference systems.
This Monday, Yuchen Zhu will present their recent paper on the FLARE framework.
Title: FLARE: Diffusion for Hybrid Language Model
Meeting Link: click here
Time: Aug 10 (Monday) 1pm ET / 10am PT / 7pm CET / 10:30pm IST
Paper: [2606.01774] FLARE: Diffusion for Hybrid Language Model
Prior knowledge:
Fundamentals of discrete diffusion (video by Sasha Rush)
Abstract:
Autoregressive (AR) large language models (LLMs) have achieved broad practical success, but sequential decoding remains a key bottleneck for low-latency deployment. Recent efficient-inference work has progressed along two axes: reducing the cost of each model invocation through efficient architectures, and reducing serial decoding steps through parallel generation. Hybrid attention backbones address the former, while diffusion language models (dLLMs) pursue the latter via iterative parallel denoising. Combining these advantages remains challenging: AR-to-dLLM conversion often fails to preserve seed-checkpoint capability, and hybrid-attention recurrent states and masking constraints make diffusion training and serving nontrivial. We present FLARE, a systematic conversion framework for hybrid-attention LLMs. Our analysis identifies transfer data quality as the primary determinant of capability preservation, outweighing loss formulation and attention-mask design. The resulting framework combines a token-equal AR-and-diffusion objective, hardware-aware kernels, and unified inference, enabling one checkpoint to support both AR-style verified decoding and diffusion-style parallel denoising. Starting from strong AR checkpoints with limited post-training data, FLARE is competitive with leading open-source dLLMs across model scales and delivers consistent throughput gains over open-source dLLM baselines in single-GPU concurrent serving. Our results further suggest that practical dLLMs are limited not only by decoding algorithms, but also by transfer data quality and the training inefficiency of current block-diffusion objectives, motivating joint design of data, objectives, architectures, and inference systems.
Yours truly,
Subham, Justin, Zhihan
Website, Twitter, Discord, YouTube
Diffusion LLM
Gentle reminder: See you all at 1pm ET / 10am PT / 7pm CET / 10:30pm IST
Meeting Link: click here
Today's paper: [2606.01774] FLARE: Diffusion for Hybrid Language Model