We would like to invite you to attend the upcoming AI Horizons talk this Wednesday (17th April, 2024)
https://rycolab.io/authors/ryan/
Date: Wed, 17th April 2024
Time: 14:00 PM - 15:30 PM
>> Kindly register for the event here:
https://forms.office.com/r/D07crNZLqm In-person Attendance
Location: Seminar Room 14, COM3-01-23, School of Computing, NUS
Live Virtual: Zoom link:
https://nus-sg.zoom.us/j/81556311003?pwd=SytCRlJLOWtlcWUyWWxTT3V6VlJMQT09 Passcode: 350204
Abstract: Language models—especially the large ones—are all the rage. And, for what will surely be one of only a few times in history, my field, natural language processing, is the center of world attention. Indeed, there is nearly a daily stream of articles in the popular press on the most recent advances in language modeling technology. In contrast to most of these articles (and most other talks on the topic), this tutorial-style presentation is not about forward progress in the area. Instead, I am going to take a step back and ask simple questions about the nature of language modeling itself. We will start with the most basic of questions: From a mathematical perspective, what is a language model? Next, the talk will turn philosophical. With all the talk of artificial general intelligence, what can theory of computation bring to bear on the computational power of language models? The talk will conclude with a statement of several recent theorems proven by my research group, the highlight of which is that no Transformer-based language model is Turing complete and, thus, we should be careful about labeling such language models, e.g., GPT-4, as general-purpose reasoners.
Bio: Ryan Cotterell is an assistant professor of computer science at ETH Zürich in the Institute for Machine Learning. He primarily works on computational and statistical approaches to language, but, every once and a while, a machine-learning paper slips in. You can read more about his and his collaborators’ research at
rycolab.io.