[Advertisement-in-Retrieval-Augmented-Generation] Task Baselines and Submission Instructions Released

[Sebastian Heineking]

Hi all,

we have updated the web page of the Advertisement in Retrieval-Augmented Generation Task:
https://touche.webis.de/clef25/touche25-web/advertisement-detection.html

Under "TIRA Tutorial and TIRA Baselines" you can find links to the submission platform TIRA that we use for this task.
Furthermore, we linked a repository with baselines that illustrate how a submission:

1. receives the input data
2. needs to format the output data
3. will be evaluated

For sub-task 1, the baseline produces a JSONL-file of generated responses. The TIRA evaluator checks if the output format is valid, i.e. contains the correct fields.
Additionally, you can use the MiniLM baseline classifier to check how many advertisements are detected.

For sub-task 2, the baselines produce a JSONL-file of predictions (Does a response contain an ad?). The TIRA evaluator checks the output format and calculates the classification effectiveness.

All of this is explained in more detail in the READMEs of the respective baselines.

Please contact us in case of any questions or issues you encounter.
We look forward to your submissions and wish you a nice weekend.

Best regards,
Sebastian

--
Sebastian Heineking

Leipzig University
Intelligent Language Technologies
Augustusplatz 10, 04109 Leipzig

sebastian...@uni-leipzig.de