Lecture on LLMs in language and text research by Andres Karjus

[Language Evolution and Learning in Amsterdam]

Dear all,

On the 22nd of October 12:00 to 13:00, Andres Karjus (Tallinn University and Estonian Business School) will give a talk in our Language Evolution and Learning Amsterdam series (see title and abstract below). Come and see the talk in person in PCH room 6.31, or join online on Zoom (Meeting ID:  820 9076 9012).

We hope to see you all there!
Kind regards,
Katrin Schulz, Raquel Alhama, Fausto Carcassi, Marieke Schouwstra

Info: https://sites.google.com/view/lela-amsterdam
Email list (sign up): https://groups.google.com/g/lelams/

***
speaker: Andres Karjus (Tallinn University and Estonian Business School)
title: Scaling the scientist: large language models in language and text research.
location: PCH 6.31. We will offer this as a hybrid event; meeting Meeting ID:  820 9076 9012

abstract:
The increasing capacities of instructable multimodal large language models (LLMs) have presented an unprecedented opportunity to scale up data analytics in sciences dealing with language, text and visual data, and to automate qualitative tasks previously typically allocated to human labor. Of particular interest to the humanities and social sciences is the capacity to use them as zero-shot classifiers and inference engines. While classifying texts or images for various properties has been long available in the form of supervised learning, the necessity to train (or tune pretrained) models on sufficiently large sets of labeled examples complicates their adoption in research beyond generic tasks like sentiment analysis, where prepackaged solutions are often available. Approaches like word or sentence embeddings and topic modeling allow for explorative approaches but are typically laborious to interpret and difficult to use for confirmatory inference. This talk discusses recent research on using LLMs in zero-shot classification and reasoning scenarios, their feasibility as replacement for typical distant reading toolsets, and potential pitfalls. LLM outputs naturally contain errors (as does human annotation), but the error rate can and should be included in subsequent statistical modeling. A bootstrapping approach is discussed, which in turn can be easily integrated in a quantitizing mixed methods research design, advocated for here as one particularly fit for purpose framework for leveraging machine scalability while supporting replicability and transparency.

[Language Evolution and Learning in Amsterdam]

Dear all,

A reminder of the next LELA talk tomorrow at 12:00 in PCH 6.31!

Kind regards,
Katrin Schulz, Raquel Alhama, Fausto Carcassi, Marieke Schouwstra

***
speaker: Andres Karjus (Tallinn University and Estonian Business School)
title: Scaling the scientist: large language models in language and text research.
location: PCH 6.31. We will offer this as a hybrid event; meeting Meeting ID:  820 9076 9012

abstract:
The increasing capacities of instructable multimodal large language models (LLMs) have presented an unprecedented opportunity to scale up data analytics in sciences dealing with language, text and visual data, and to automate qualitative tasks previously typically allocated to human labor. Of particular interest to the humanities and social sciences is the capacity to use them as zero-shot classifiers and inference engines. While classifying texts or images for various properties has been long available in the form of supervised learning, the necessity to train (or tune pretrained) models on sufficiently large sets of labeled examples complicates their adoption in research beyond generic tasks like sentiment analysis, where prepackaged solutions are often available. Approaches like word or sentence embeddings and topic modeling allow for explorative approaches but are typically laborious to interpret and difficult to use for confirmatory inference. This talk discusses recent research on using LLMs in zero-shot classification and reasoning scenarios, their feasibility as replacement for typical distant reading toolsets, and potential pitfalls. LLM outputs naturally contain errors (as does human annotation), but the error rate can and should be included in subsequent statistical modeling. A bootstrapping approach is discussed, which in turn can be easily integrated in a quantitizing mixed methods research design, advocated for here as one particularly fit for purpose framework for leveraging machine scalability while supporting replicability and transparency.