Pushing the Limits of Sparse Attention: From Theory to Practical Efficiency - Marcos Treviso (Instituto de Telecomunicações)

[Gonçalo Correia]

Dear all,

We’re excited to invite you to the next Priberam Lunch Seminar, featuring Marcos Treviso (Postdoctoral Researcher at Instituto de Telecomunicações). Marcos will present recent advancements in adaptive sparse attention, a promising alternative to dense attention in transformers.

His talk will cover key insights into the expressivity, generalization, and hardware efficiency of sparse attention mechanisms, including:

🔹 The expressivity of sparsemax and entmax attention compared to dense attention.
🔹 How sparse attention improves generalization for long sequences.
🔹 AdaSplash – a new hardware-aware implementation that outperforms FlashAttention-2 at high sparsity.

📅 Date: Tuesday, April 8th
⏰ Time: 1 PM
🏢 In-person: Instituto Superior Técnico, room PA2 (Alameda Campus)
💻 Online: Zoom - https://us02web.zoom.us/j/89922277557

🥪 Lunch bags will be provided for in-person attendees.

📌 Register here (mandatory for in-person attendence):
https://www.eventbrite.pt/e/pushing-the-limits-of-sparse-attention-from-theory-to-practical-efficiency-tickets-1291160538929

Additionally, the video of our last seminar with João Gante is now available on YouTube! 🎥 If you missed it or want to revisit the discussion on recent LLM and VLM advancements, you can watch it here:

https://www.youtube.com/watch?v=nrVk3a5lKuE

Looking forward to seeing you at the next session!

Best,
Gonçalo