Lecture on questions by Polina Tsvilodub
5 views
Skip to first unread message
Fausto Carcassi
Jun 10, 2024, 5:21:26 AM6/10/24
to Language Evolution and Learning in Amsterdam
Dear all,
Next Tuesday 13:00-15:00, Polina Tsvilodub (University of Tübingen) will give a talk in our Language Evolution and Learning Amsterdam series (see title and abstract below). Come and see the talk in person in PCH room 4.11, or join online on Zoom (https://uva-live.zoom.us/j/5046349544, Meeting ID: 504 634 9544).
We hope to see you all there!
Kind regards,
Katrin Schulz, Raquel Alhama, Fausto Carcassi, Marieke Schouwstra
Email list (sign up): https://groups.google.com/g/lelams/
p.s. Our email list is still modest in size; if you think of anyone who might be interested in this talk, please forward this message!
***
speaker: Polina Tsvilodub (University of Tübingen)
title: How to be relevantly overinformative to a polar question: Reasoning about questioner goals to provide more relevant answers
location: PCH 4.11. We will offer this as a hybrid event; meeting Meeting ID: 504 634 9544
abstract:
Imagine you are working as a barista at a coffeeshop. A customer asks a polar question like “Do you have iced tea?” but you’ve run out. In this situation, you might likely provide an overinformative answer going beyond a simple “yes” or “no” (e.g., “No, but we’ve got iced coffee!”), but what principles guide the selection of additional information?
This talk proposes that such answers draw on learning about our interlocutors from language; they present a non-trivial instance of pragmatic communication which depends for complex reasoning drawing on these inferences about the interlocutors and world knowledge. Specifically, I will argue that respondents use the uttered question in order to reason about the questioner’s preferences and goals, and craft answers relevant to those goals (Hawkins & Goodman, 2017). I will provide experimental evidence from several human studies suggesting that overinformativeness in human answering is driven by considerations of relevance to the questioner’s goals which respondents flexibly adjust given the functional context in which the question is uttered, and given the shared world knowledge. Furthermore, I will present a Rational Speech Act model (RSA, Frank & Goodman, 2012) of pragmatic overinformative question answering which builds on an action-based notion of relevance of information. It captures qualitative patterns in speakers’ utterance choices across a variety of contexts.
Finally, I will present a comparison of these human and probabilistic modeling results to question-answering performance of a variety of state-of-the-art neural language models. We find that most models fail to adjust their answering behavior in a human-like way and tend to include irrelevant information (Tsvilodub et al., 2023). We show that most recent fine-tuned LLMs are highly sensitive to the form of the prompt and only achieve human-like answer patterns when guided by an example of a relevantly overinformative answer or a cognitively-motivated explanation of the complex reasoning.
This talk proposes that such answers draw on learning about our interlocutors from language; they present a non-trivial instance of pragmatic communication which depends for complex reasoning drawing on these inferences about the interlocutors and world knowledge. Specifically, I will argue that respondents use the uttered question in order to reason about the questioner’s preferences and goals, and craft answers relevant to those goals (Hawkins & Goodman, 2017). I will provide experimental evidence from several human studies suggesting that overinformativeness in human answering is driven by considerations of relevance to the questioner’s goals which respondents flexibly adjust given the functional context in which the question is uttered, and given the shared world knowledge. Furthermore, I will present a Rational Speech Act model (RSA, Frank & Goodman, 2012) of pragmatic overinformative question answering which builds on an action-based notion of relevance of information. It captures qualitative patterns in speakers’ utterance choices across a variety of contexts.
Finally, I will present a comparison of these human and probabilistic modeling results to question-answering performance of a variety of state-of-the-art neural language models. We find that most models fail to adjust their answering behavior in a human-like way and tend to include irrelevant information (Tsvilodub et al., 2023). We show that most recent fine-tuned LLMs are highly sensitive to the form of the prompt and only achieve human-like answer patterns when guided by an example of a relevantly overinformative answer or a cognitively-motivated explanation of the complex reasoning.
Reply all
Reply to author
Forward
0 new messages