CLEF 2023 LongEval Lab on Longitudinal Evaluation of Model Performance

[Arkaitz Zubiaga]

Call for Participants

LongEval: Longitudinal Evaluation of Model Performance
CLEF 2023 Lab

https://clef-longeval.github.io/

18-21 September, Thessaloniki, Greece

--------------------------------------------------------

** Registration open, training data available **

The CLEF 2023 LongEval lab is motivated by recent research showing that
the performance of information retrieval and text classification models
drops as the test data becomes more distant in time from the training
data. LongEval differs from traditional IR and classification shared
tasks with special considerations on evaluating models that mitigate
performance drop over time. It encourages participants to develop
temporal information retrieval systems and longitudinal text classifiers
that survive through dynamic temporal text changes, introducing time as
a new dimension for ranking models performance.

The lab consists of two tasks:
* Task 1. LongEval-Retrieval: The goal of Task 1 is to propose an
information retrieval system which can handle changes over the time. The
proposed retrieval system should follow the temporal timewise evolution
of Web documents. Contact: longeval...@univ-grenoble-alpes.fr

* Task 2. LongEval-Classification: The goal of Task 2 is to propose a
temporal persistence classifier which can mitigate performance drop over
short and long periods of time compared to a test set from the same time
frame as training. Contact: r.a.a.a...@qmul.ac.uk

Registration
------------
Please use the CLEF registration form
(http://clef2023-labs-registration.dei.unipd.it/) to register to the
tasks. The registration closes on Friday, 28 April 2023.

Timeline
--------
April 2023: Test data release
28 April 2023: Registration closes
10 May 2023: End of Evaluation Cycle [submission of runs]
5 June 2023: Submission of Participant Papers [CEUR-WS]
5–23 June 2023: Review process of participant papers
July 2023: Camera ready paper submission
September 2023: CLEF Conference

Organisers
----------
Rabab Alkhalifa, Iman Bilal, Hsuvas Borkakoty, Jose Camacho-Collados,
Romain Deveaud, Alaa El-Ebshihy, Luis Espinosa-Anke, Gabriela
Gonzalez-Saez, Petra Galuščáková, Lorraine Goeuriot, Elena Kochkina,
Maria Liakata, Daniel Loureiro, Harish Tayyar Madabushi, Philippe
Mulhem, Florina Piroi, Martin Popel, Christophe Servan, Arkaitz Zubiaga.