Fwd: [security-lunch] (Oct 14) "Securing LLMs Against Prompt Injection for Agentic Applications" (Sizhe Chen)

[Alan Karp]

--------------
Alan Karp

---------- Forwarded message ---------
From: Ihyun Nam via security-lunch <securit...@lists.stanford.edu>
Date: Mon, Oct 12, 2026 at 9:37 AM
Subject: [security-lunch] (Oct 14) "Securing LLMs Against Prompt Injection for Agentic Applications" (Sizhe Chen)
To: securit...@lists.stanford.edu <securit...@lists.stanford.edu>

Security Lunch: Wednesday, October 14th, 2026, 12:00PM @ CoDa E160

Title: Securing LLMs Against Prompt Injection for Agentic Applications

Speaker: Sizhe Chen (UC Berkeley)

Can't make it in person? Join us on zoom: https://stanford.zoom.us/j/92492090915?pwd=gpobdILCkAPS6UbsRg86y8NhvFuDmn.1.
See our past and upcoming events on our website: https://securitylunch.stanford.edu/.

Abstract: Prompt injection is widely recognized as a major security threat to AI agents, in which external untrusted data (websites, documents, and emails) may contain an injected prompt (“Ignore previous instructions and …”) that arbitrarily manipulates an agent’s operations. The best-defended existing LLMs still suffer from near-100% attack success rates (ASRs) under adaptive prompt-injection attacks. In this talk, I will discuss our latest efforts, SecPO and SecOPD, to fine-tune LLMs with an order-of-magnitude lower ASRs (0% -- 12.3%) against the current strongest prompt-injection attacks. Specifically, we (1) separate the trusted prompt and untrusted data into two input channels; (2) simulate strong adaptively attacked training samples for the model to learn robustness from; and (3) design the loss according to a principle that approximates a (nonexistent) secure-and-powerful LLM. Without a noticeable utility drop, our defended Qwen3.6-27B achieves security that generalizes to realistic agentic tasks with multiple trusted and untrusted messages. We fully open our training set with optimized injections, code for our recipe, and the defended model, which is securer than GPT-5.5, the frontier defense.

Bio: Sizhe Chen is a Computer Science Ph.D. candidate at UC Berkeley’s Berkeley AI Research (BAIR), working with Prof. David Wagner. He studies real-world AI security problems with general and principled solutions. He developed StruQ, Meta-SecAlign, SecPO, and SecOPD, the most robust open LLMs against adaptive prompt injection attacks. His research has been supported by selective industry fundings, including NVIDIA Fellowship, Meta-BAIR Commons, and Google-BAIR Commons. Previously, he obtained an M.Eng. and a B.Eng. from Shanghai Jiao Tong University. https://sizhe-chen.github.io

_______________________________________________
security-lunch mailing list
securit...@lists.stanford.edu
https://mailman.stanford.edu/mailman/listinfo/security-lunch