Stanford MLSys Seminar Episode 39: Laurel Orr [Th, 1.35-2.30pm PT]

[Karan Goel]

Hi everyone,

We're back with the thirty-ninth episode of the MLSys Seminar on Thursday from 1.35-2.30pm PT. 

We'll be joined by Laurel Orr, who will talk about building infrastructure for training foundation models. The format is a 30 minute talk followed by a 30 minute podcast-style discussion, where the live audience can ask questions.

Livestream Link: https://www.youtube.com/watch?v=g-OjU4uzWqE

Guest: Laurel Orr

Title: Towards Transparent Foundations -- Building Accessible Infrastructure for Training Large-Scale Language Models

Abstract: “Foundation models” — large-scale self-supervised models that can be adapted to a wide range of downstream tasks - are changing how machine learning systems are constructed and deployed. Due to their extreme resource demands, training and developing a science behind these models has remained difficult. In this talk, I'll introduce and describe the journey behind Mistral, an infrastructure for accessible, easy-to-use foundation model training. I'll describe some of the hurdles we encountered with stable, reproducible training and how we see Mistral as a crucial step to facilitate open foundation model research.

Bio: Laurel Orr is currently a postdoc at Stanford working with Chris Ré in the Hazy Research Lab. In August of 2019, she graduated with a PhD from Paul G Allen School for Computer Science and Engineering at the University of Washington in Seattle. She was part of the Database Group and advised by Dan Suciu and Magdalena Balazinska. Her research interests are broadly at the intersection of machine learning and data management. She focuses on how to manage the end-to-end lifecycle of self-supervised embedding pipelines. This includes problems of how to better train, maintain, monitor, and patch the embedding models and their use downstream.

See you all there!

Best,

Karan

[Karan Goel]

Reminder: this is in 15 minutes!