Task Rules Clarification

[SemEval 2026 Task 5 Participants]

Hello,

We recently received a question about the use of decimal numbers as predictions. While we originally intended for the task to be on an ordinal scale (based on integer values), some teams may have already developed their systems assuming that decimal numbers are allowed, since there were no rules about that. Therefore, we are writing to officially inform everyone that decimal predictions are allowed for this task. In fact, we explicitly encourage the use of decimal scores for the reason outlined below.

The use of decimal predictions impacts evaluation metrics, specifically the Accuracy w/in Standard Deviation metric. By definition, the acceptable “correct” range around the average cannot be smaller than 1 (exclusive). As a result, predictions at the extremes of the scale (1 and 5) are inherently disadvantageous because they span a smaller portion of the scale. For this reason, regression-based models might be at an advantage over classification models when it comes to task performance.

We are nevertheless interested in seeing both classification and regression models used on our benchmark.

The evaluation phase still runs until the end of January. Good luck!