Reading Group Meeting #197: In-context learning in presence of spurious correlations

[Tatev Vardanyan]

Hi friends, 

We are glad to invite you to tomorrow's reading group meeting. See details below:

Samvel Karapetyan, a researcher at YerevaNN and a second-year undergraduate student at YSU's Information and Applied Mathematics department, will present a paper that one of our teams submitted to the NeurIPS conference yesterday.  

The paper is about In-context learning in presence of spurious correlations. It discusses how large language models exhibit a remarkable capacity for in-context learning, where they learn to solve tasks given a few examples. Recent work has shown that transformers can be trained to perform simple regression tasks in-context. This work explores the possibility of training an in-context learner for classification tasks involving spurious features. We propose a novel technique to train such a learner for a given classification task. 

Remarkably, this in-context learner matches and sometimes outperforms strong methods like ERM and GroupDRO. However, unlike these algorithms, it does not generalize well to other tasks. We show that it is possible to obtain an in-context learner that generalizes to unseen tasks by constructing a diverse dataset of synthetic in-context learning instances.

See you tomorrow at the YSU Krisp-AI Lab at 4:30 PM. 

Regards,

Tatev