AdabEval 2026: Shared Task at OSACT7 (LREC 2026)
شريفة الغامدي
The website for the shared task is: https://sites.google.com/view/adabeval2026/home
Description:
Participants will develop systems to perform one of the following two subtasks, which investigate politeness in Arabic social media posts:
Politeness Classification (Subtask A):
This subtask focuses on building and evaluating models that automatically assess the politeness level of Arabic social-media posts. Given a post’s text, participants must classify it into one of three categories: Polite, Neutral, or Impolite. Systems will be compared using accuracy as well as macro-averaged precision, recall, and F1-score on the test data.
Subtask A Summary
- Input: Arabic post
- Output: One of 3 predefined category labels (polite, neutral or impolite)
- Metric: Accuracy, macro-averaged precision, recall, and F1-score
Category Prediction (Subtask B):
In this subtask, the goal is to evaluate the ability of systems to identify multiple pragmatic functions in Arabic social-media posts. Each text may express one or more categories from nine culturally grounded functions such as criticism, insult, respect, prayer, greeting, or hospitality. The task is framed as multi-label classification, and systems must predict all applicable categories for each instance. The official evaluation metric is macro-averaged F1-score across the nine categories.
Subtask A Summary
- Input: Arabic post
- Output: All applicable categories for each instance.
- Metric: Macro-averaged F1-score
Dataset Information:
The dataset is an annotated corpus of Arabic social‑media posts. Each record contains an identifier, raw text, a primary label (polite, neutral or impolite), and up to three category-keyword pairs. Categories capture pragmatic functions such as Criticism, Insult, Disparagement, Prayers, Greetings, Admiration, Respect, Felicitation and Hospitality & generosity. Keywords highlight words or phrases that motivated the annotation.
EXAMPLE 1:
---------------
Sentence: مبدعون ومميزين بطرحكم وضيوفكم. جزاكم الله خير
الجزاء
Source: YouTube
Label: Polite
Criteria 1: Appreciation & Love (الإعجاب والحب) — Keywords: مبدعون،
مميزين
Criteria 2: Thanks & Gratitude (الشكر والامتنان) — Keywords: جزاكم
الله خير الجزاء
EXAMPLE 2:
---------------
Sentence: من كان بتخيل دكتور معتز ومقالاته قبل ثوره ٢٥
يناير يتحول لحذاء في أيدي العسكر ويصبح مجرد دلدول ويتبع اُسلوب منحط ووضيع كده
Source: Tweet
Label: Impolite
Criteria 1: Insult (شتيمة) — Keywords: منحط،
وضيع
How to Participate?
1. Registration is required, please complete the
registration form.
2. Join the AdabEval2026
at slack.
System Description Papers
All participating teams are encouraged to submit a short system description paper. Papers will be included in the workshop proceedings and do not require high leaderboard ranking. We welcome creative approaches, analysis, and lessons learned.
Contact
For questions or clarifications, please contact the organising team at adabev...@gmail.com.
We look forward to your participation and contributions!
Best regards,
The AdabEval 2026 Organizing Team