Using Algorithms to Understand Transformers (and Using Transformers to Understand Algorithms) -- Vatsal Sharan (USC)

[Hongyang R. Zhang]

Hi everyone,

I hope you've been doing well.

Next Thursday, Oct 23, Vatsal Sharan (USC) is visiting around the Boston area and he reached out to me expressing his interest in giving a talk at Northeastern. I am forwarding his talk title and abstract in the message below.

The talk will be held at 177-2206 (and online via Zoom), from 3pm to 4pm -- if you want to attend in-person, please let me know.

Title: Using Algorithms to Understand Transformers (and Using Transformers to Understand Algorithms)

Abstract: We will discuss how algorithmic tools and understanding borrowed from optimization theory, Fourier transforms, and Boolean function analysis can help understand the mechanisms employed by Transformers to solve basic computational tasks such as linear regression and addition. We will examine the role of the architecture and pre-trained data in enabling Transformers to learn their employed mechanisms. Finally, we will discuss work on using Transformers themselves to discover and design data structures for tasks such as nearest neighbor search.

Thanks,

Hongyang