Session 2 // Nov 24: Diffusion beats AR: Controllable Generation

[Diffusion LLM]

Hello folks,

Discrete diffusion models offer much greater control over the generation process, making them a compelling alternative to autoregressive models.

This Monday, Sophia Tang and Pranam Chatterjee will explain why and demonstrate how discrete diffusion models enable more controllable generation.

Title: PepTune: De Novo Generation of Therapeutic Peptides with Multi-Objective-Guided Discrete Diffusion

Meeting Link: click here

Time: Nov 24 (Monday) 10 am ET / 4pm CET 

Paper: https://arxiv.org/abs/2412.17780

Prior knowledge: Fundamentals of discrete diffusion

Abstract: We present PepTune, a multi-objective discrete diffusion model for simultaneous generation and optimization of therapeutic peptide SMILES. Built on the Masked Discrete Language Model (MDLM) framework, PepTune ensures valid peptide structures with a novel bond-dependent masking schedule and invalid loss function. To guide the diffusion process, we introduce Monte Carlo Tree Guidance (MCTG), an inference-time multi-objective guidance algorithm that balances exploration and exploitation to iteratively refine Pareto-optimal sequences. MCTG integrates classifier-based rewards with search-tree expansion, overcoming gradient estimation challenges and data sparsity. Using PepTune, we generate diverse, chemically-modified peptides simultaneously optimized for multiple therapeutic properties, including target binding affinity, membrane permeability, solubility, hemolysis, and non-fouling for various disease-relevant targets. In total, our results demonstrate that MCTG for masked discrete diffusion is a powerful and modular approach for multi-objective sequence design in discrete state spaces.

Yours truly,

Subham, Justin, Zhihan

Website, Twitter, Discord, YouTube

[Diffusion LLM]

Just a reminder that the talk is happening now!

====================================

[Diffusion LLM]

Hello folks, the recording of Sophia's talk is now available on YouTube, make sure to check it out: https://www.youtube.com/watch?v=NX2Mth_Epvk